The driving principle behind development here at AidScholar has been Customer Development (and partially the lean startup idea from Eric Reis). Essentially, customer development is the idea that new startups know nothing. Everything is an unknown – your user, the user’s problems, and the right solution. The job of the startup is to measure and validate any ideas with potential customers before moving on and scaling up.
The initial problem
I’ve always felt that it’s a serious PITA to find scholarships online. The only two options are general web search (Google) and wizard-based matching services (Scholarships.com, FastWeb).
With general web search, most results are usually high ranking wizard-based matching sites or articles about scholarships. Usually you won’t find many individual scholarships in general web search.
On the other hand, wizard-based matching services return individual scholarships, but you never know if they are returning all scholarships available. All of these services force you to fill out a long questionnaire before returning scholarships, but what if you aren’t sure what category to pick? Should you select “journalist” as a future career to retrieve scholarships for literary students? Or will that only return scholarships for those majoring in journalism? So you are always second guessing your choices as you fill out the form. Am I getting the most comprehensive list or should I be entering some special term to get it? Going back to refill the form again to access more results gets to be tedious and can be very frustrating. It becomes a “luck of the draw” effort where a person can be crossing his or her fingers in the hopes that the terms entered will be the magic key to unlock the doors of the scholarship world. And we haven’t even address the fact that the terms are of a limited set.
So with that in mind, I figured to solve the problems associated with finding scholarships online, users wanted a fast and easy search engine tailored to return individual scholarships. A tool that a user can utilize with the confidence that the results are both specific and comprehensive.
Now I just needed to validate that idea.
Minimum viable product – validating the idea
I started off by building what I thought the Minimum Viable Product would be – a simple search box that returned scholarship results.
I sent this really rough demo around to a bunch of people to get their thoughts. Generally most people thought the demo was rough, but the idea was solid, they could see the potential and might use it given some changes.
Version 1
Now that I had some small validation, I was more motivated to continue building the product. Most of my feedback had a reoccurring theme: people wanted a scholarship search site that was fast, easy to use, and comprehensive.
I broke that down in to three features:
- For speed, I’d use ajax to make it seem like the site was more responsive
- For easy of use, I added refinement options and related searches
- For comprehensiveness, I tried to aggregate as much information possible
This led to a couple different versions:
I eventually settled on the version with less features (I stripped the tags, related searches, key phrases) because they didn’t really deal with the core product idea.
Feedback Round 2
Now that I had a working product, I went ahead with the product launch.
This time around, I asked for customer feedback in more places. I setup a Twitter account and started contacting people for their thoughts. I setup this blog. I went back to everyone from my last release and asked for their feedback. I paid for a couple user reviews on UserTesting.com. I learned alot from these indepth / interactive interviews, but I still only had about a dozen user interviews. I needed to reach more people.
Fast surveys on the cheap
In order to reach more people, I setup a survey through Amazon’s Mechanical Turk. In total, I paid $4.40 for 40 responses.
Here’s the task/survey I created:
Task
Background: You are a student looking for a scholarship to apply to.
1. Visit http://aidscholar.com/
2. Search for a scholarship
3. Browse through the scholarship list.
4. Click through to the individual scholarship’s site.Answer a short survey.
Hello, I’m collecting feedback on my website. Your answers let me know if my site is sending the right message.
1. What does this site do?
2. Did you find what you were looking for?
3. What did you like about this site?
4. What didn’t you like about this site?
5. What would you change?
6. What features would you like to see?
7. Would you recommend this to a friend?
8. Please provide any comments you may have below, we appreciate your input!
I analyzed the survey responses, and came up with the following:
Summary
- 34 out of 40 people would recommend site to a friend.
- 3 out of 40 people might recommend site to a friend.
- 3 out of 40 people would not recommend site to a friend.
Things ppl liked:
- Database size
- Simple design
- Refinement options
- Ease of use
- No registration needed
- Speed / responsiveness of site
Things ppl want changed / not liked:
- Sort feature
- Tips / User Guide
- Design / Not professional look
- Accuracy / Comprehensiveness of search engine
- Label naming
- More meta data about scholarship before leaving site
- Larger number of scholarships shown per page
- Wizard functionality
While Mturk might not be my exact target market, many of the responses actually came from users that did fit.
Here’s a comment I got from someone in the process of looking for scholarships:
At present I am looking for scholarships and this site is a great option to continue my search.
And here’s another comment from someone with experience with competing products:
Great idea. As the parent of a soon to be (3 days) HS graduate, I have looked at a lot of these sites. I like yours.
Overall I felt the learnings I gained from Mturk were amazing compared to the time/cost to run the survey.
So I learned that in general, my initial hypothesis that people wanted a fast, easy to use, comprehensive scholarship search site was spot on. Even though I wasn’t able to deliver completely with the product, most people would recommend AidScholar to a friend. The main issues that kept coming up dealt with the design and comprehensiveness of the results.
Interestingly, several people wished that I offered a wizard-based matching feature, contrary to my initial assumption. I’m not sure if I should add the feature to the next release, but it’s definitely something to think about.
Redesign
Armed with a ton of feedback and potential feature/fixes, I went back to work on the next release.
The current version of AidScholar had 3 distinct problems (as determined by user feedback):
- The design did not feel trustworthy
- Lacked a user’s guide or helpful tips on how to use the site
- No deadline sort feature
In order to address the design issues, I tried a more “open” and “clean” look with the redesign. I added a tour/overview section and created a short screencast that walks a user through the site. Along with the redesign, I added the sort feature.
Hopefully this addresses all the issues with my previous version, but we’ll have to see.
Next steps
Following this recent release, I’m back into learning mode and trying to gathering more feedback. We’d love to hear what you think about the new redesign and any other feedback you have.
Hope this helped you understand more about the process / development happening at AidScholar. We’re really trying to make AidScholar the best scholarship search site out there and I think we can with your help.
