I’m currently eagerly awaiting an email to drop into my inbox. It is the one that will tell me whether I have passed or failed my second attempt at my exam to become a web search evaluator for Leapforce (who are now part of Appen). If I pass, it will open the door to a flexible way of working that will be available whenever I want it. If I fail, I have the option to apply to Lionbridge who offer the same work. As someone who spends a lot of time on the internet, this type of work should suit me really well. In this post I explain more about what the role involves and why this job will be perfect for me as a working mum.
A web search evaluator assesses the search results, webpages and adverts generated by search engines. They look for how high the quality and suitability is for an everyday user. The feedback given from the evaluation helps to make search engines more powerful and to improve people’s experiences of using the internet.
The companies offering this work recruit evaluators from all over the world. An internet user from the UK will have very different expectations to someone from the USA, or Spain, or Australia. Recruiting users local to each country ensures the evaluations are as useful as possible.
I originally applied to Leapforce in early June. There are multiple stages to get through before you actually become an evalutor for them.
First, I filled out a form online and submitted my CV. If you apply to any of the companies for this work I recommend treating it like a proper job application. Tailor your CV to make it look appealing. If you have examples of using research, critical thinking or analytical skills make sure you mention these.
My application was accepted and I was invited to sign some documents to confirm that I wanted to join Leapforce’s panel. I was put in a pool of potential evaluators for 2 different projects. I then had a couple of months to wait before I heard from Leapforce again.
In August, I was scheduled to take an exam in order to join one of the projects after vacancies opened up for it. The exam date didn’t suit me, but I was easily able to move it to a different date. You can do this multiple times as long as you reschedule before the day you are due to start the exam.
In the email inviting me to take the exam I was given a link to an online version of the guidelines for the web search evaluation role. I read through these before my exam start date. There was a LOT to read and I underestimated how long it would take me. I felt like I understood it all OK, there was just a lot to remember.
I was given 7 days to complete the exam. I thought this would be loads of time but I ended up needing all of it! Leapforce provided me with a link to download the PDF copy of the guidelines. I was allowed to refer to these throughout the exam and I needed to! It was difficult to remember all the information after only reading it one time so far.
There were 3 parts to the exam. A theory test, assessing the quality of 7 different webpages, and assessing 7 different sets of search results. I had to pass the theory test to get through to the next 2 stages. The webpages I needed to look at all needed to be assessed from the point of view of a mobile phone user. Although I was viewing the webpages on my phone, I was able to do the research and evaluation about them on my laptop.
The theory test took me 4 hours. The assessments took me about an hour per webpage / search result set. So I definitely needed the full week to be able to complete the exam!
I had to wait until 1 week after the start date of my exam to receive my results. The email arrived just after 5pm. I found out I had failed which was really disappointing after all the time and effort I had put in. Unfortunately I did not receive any feedback about my results, only the offer to retake the exam. I scheduled a new date for it and waited for my chance to take it again.
Before taking the exam again I reread the guidelines. Having had the opportunity to actually do the assessments in the first exam I felt I understood the guidelines much better.
The second time round the theory test took me only 2 hours to complete. The answers came to me much more easily. I passed the test and went on to do the page and search result assessments.
Again I felt I understood what I was doing a lot better. I didn’t have to refer to the guidelines as intensely as I did the first time round. I got all 7 of the page quality assessments done on day 2 of the exam. On day 3 I was able to finish all 7 of the search result assessments.
Tips for the Exam
I can’t reveal too much about the guidelines or the exam content, but these general tips should be useful.
The exam is bloody hard. Make sure you leave yourself enough time to do it. Read the guidelines twice before attempting the exam. Reschedule the exam if you need more time to read them.
Reread the second part of the guidelines (about page quality ratings) before completing the page quality assessments. Carefully study the examples they give you in the guidelines.
Reread the third part of the guidelines (about the users’ needs being met) before completing the search result assessments. Again, make sure you carefully study the examples.
Be very cautious with giving the highest ratings. Make sure the page or result meets ALL the criteria. It is rare for a page or result to warrant the highest rating.
The Retake Results
I passed! So it was definitely worth making the effort to try again. The next steps are for Leapforce to approve another set of documents that I have signed. This will enable me to become an independent contractor for them. I also need to set up my computer according to their requirements. They have sent me an email telling me what I need to do.
Thank you for reading!
I hope this has given you an insight into the web search evaluation application process. Have you gone through the process yourself for Lionbridge? It would be great to hear about your experiences in the comments below! To find out what it has been like during my first few days of trying the job out, read my follow up post here.
Subscribe to New Posts