Frequently asked questions

Getting Started:

How can I access the list of defects that were reported?

When logged in, select your project. You’ll see a button labelled: « view bug base » link for functional testing projects or a “view test results” link for UX testing projects.

What about confidentiality?

  • All of our permanent employees have signed a formal confidentiality agreement that covers all of your projects.
  • All of our freelance testers must agree to keep your projects’ information confidential before they can apply to work with us.
  • Only the testers who are involved in testing your projects have access to any login or design information that you may provide.
  • We closely monitor the work of our community of freelance testers. We know who works on every project, and if there’s ever any doubt over a person’s actions with regards to confidentiality, they are immediately banned.

 

What is crowd testing?

Crowd testing consists of delegating software testing tasks to a community of expert quality assurance testers through an online platform. It offers many advantages:

  • Flexibility: Given that our crowd encompasses testers in all timezones, there are always testers available, no matter the time of day. This comes in particularly handy when working late at night trying to meet a deadline.
  • Instantaneity: Unlike traditional software testing companies, crowd testing services are available on short notice and allow you to get results in as little as a few hours.
  • Real-world multiplatform testing: When dealing with external users, your products are likely to be used on a much wider range of hardware and software combinations than those you have in-house. These include older browser versions, other operating systems and different hardware devices. By working with a large crowd of testers, you ensure that your product is tested in various real-world situations and on a large variety of hardware and software combinations.

 

Process:

How quickly can tests begin?

Tests can begin quickly if you have adequate documentation ready. In a perfect world, we’d hopefully have enough time to review your test objectives and documentation before it is sent to our testers. The more time you have to prepare, the better, but we can also work quickly if you need it!

How to work with your team in Lean Testing

Once your list of defects has been exported to Lean Testing, your developers can get to work.

Lean Testing’ bug sorting system allows them to first filter defects by type of issue or severity and determine which ones they should fix first.

Developers can change bugs status to distinguish the defects that have already been fixed, and the ones that remain to be corrected. This feature is very useful since it allows them to see how close they are to completion.

If your developers need more information to understand specific bugs, they can communicate with the team members using the conversation space below “bugs descriptions”.

When should I test my software?

We believe in testing as a continuous task, which should be performed alongside development. It is much more efficient for testers and developers to communicate throughout the entire development process, instead of separating testing and development and only leaving testing until the end.

The earlier you detect certain defects, the easier they will be to fix and continue along the development process.

Here is an interesting article if you want more information on whether or not you are testing early enough.

However, it is important to remember that testing does not end when you release your apps or software. In order to stay relevant today, you must have a continuous update plan, and Crowdsourced Testing can help you with this!

Once you have gained a large amount of users, it is twice as important to make sure that new updates are well-received in order to maximize user-retention. Crowdsourced Testing helps many companies organize and test their app updates to maximize user-retention and in-app purchases.

Pricing:

When do I pay?

If you are using our self-service option, you only pay when you are ready to begin the tests.

For our monthly and dedicated plans, someone from our team will contact you to arrange monthly payments via the method of your choice.

Testing Methodology:

What kind of results do I get?

Standardized and complete bug reports: to help you understand defects instantly, each bug report contains answers to specific mandatory questions, such as the steps to reproduce bugs and a definition of the expected result. Every bug report also comes with a screenshot or video.

A clear and concise list of defects: once your testing session has ended, you are invited to visit a comprehensive list including all of the defects identified in your product. You can sort out defects by type, sections where the issue happens and level of severity. To facilitate developers’ work and streamline your internal resolution process, you can export this list of defect to Lean Testing, our free hosted bug tracker, or Jira.

Can I provide test cases?

You can absolutely provide test cases that our testers will follow if you wish to do so.

If you’d like to prepare test cases beforehand but don’t know where to start, contact us and we’ll be happy to help!

We can also perform exploratory tests, during which our testers use your product as any normal user would and report any bug they find along the way.

Who are the members of your software testing community?

Testers involved in testing your projects are experienced professionals who have demonstrated their qualifications by participating in a large number of projects with us.

On average, members of our community have over 6 years of experience as full-time testers.

Our tester screening process is ongoing. When signing up, we require their CV and/or LinkedIn profile. Once they’ve registered with us and started working, we constantly review their work to confirm their seriousness, dedication and performance.

The Lead Testers involved in managing the work performed by the crowd of testers are all internal employees based in our office in Montreal, Canada. Their mission is to review our testers’ work, grade their performance and ensure that you only get valid results.

Unlike freelancer platforms, you are not required to select the individual testers who will participate in your projects. Our algorithm automatically selects the best testers for your projects, based on their performance with us, their expertise and the devices you are targeting.

If you’d like more insights into the members of our software testing community, take a look at the article: “Who are the mobile early adopters?” on our blog.

Installation & Distribution:

Can we use TestFlight / HockeyApp / Crashlytics to send the build to testers?

Absolutely! Your Project Manager can even make the arrangements for you.

On what devices can my project be tested?

We focus primarily on computers (Mac and PC), mobile devices and smartboxes like Apple TV, Roku and Samsung Smart TVs. You can specify precisely what devices you’d like to have your projects tested on, for example: iPad Mini running iOS 10 or very broad, for example: all recent popular desktop browsers.

Tester Questions:

How can I become a tester with Crowd Testing?

You just need to signup here.