[LON-CAPA-users] Fwd: Support research and win an iPod Shuffle

Gerd Kortemeyer lon-capa-users@mail.lon-capa.org
Tue, 26 Aug 2008 21:18:10 -0400

A message from Lars Hoeppner:


I'd like to thank the people who already participated in the evaluation.

However, we still need your input!

To make your participation in the evaluation a little easier, I
propose this procedure:

1) choose a search query (and a start user if necessary, i.e. if
you're not a course instructor in some course)
2) use the keyword-search (algorithm 5) and then give a rating based
on those questions:
- are most of the resources found relevant to you or is a majority not
- how long would it take you to select the best ones out of the list?
3) issue the same search query but use a filter algorithm (1-4) (keep
the default options) and give a rating based on:
- compared to the results you found with the step 2 before, do you
feel that all relevant resources are still in the result list?
- are the less relevant resources no longer in the list?
- will it take you less time to select the best resources than with
the complete list from step 2?

This usually shouldn't take you longer than a few minutes and would
already help a great deal.

Ideally, do step 3 for all four filter algorithms. If you want to go
wild, you could also try the extend-links on the result page or even
try changing the parameters.
Note regarding browsers: I recently realized that the search result
pages are "optimized" for Mozilla Firefox. The extend-links somehow
get truncated when using Internet Explorer, so in case you use IE you
can only evaluate the filter algorithms.

Thanks for your help!


On Thu, Aug 21, 2008 at 7:55 PM, Gerd Kortemeyer <korte@lite.msu.edu>  
> Hi,
> Lars Hoeppner, whom some of you met at the LON-CAPA Conference this  
> year, is
> working on his master's degree with the Fern-Universitaet Hagen in  
> Germany.
> His topic is community-supported search in an educational content  
> pool with
> sparse metadata, aka LON-CAPA.
> There are a number of possible algorithms for community-supported  
> search,
> and the last phase of his project is evaluation by users.
> As you are likely setting up your courses this week, please give it  
> a whirl.
> Login to
> http://s12.lite.msu.edu/
> using your normal usernames and password, and by hand changing the  
> "Domain"
> field to your domain. Then go to the course you want to be working  
> on and
> use Search like you normally would.
> You get a choice of mechanisms and parameters. Try out a few and  
> please
> evaluate the search results using the
> How well do the results meet your expectations?
> (bad) 1 2 3 4 5 (good)
> field at the top. Please don't forget to "Send".
> Note that the time it takes for these searches is not a valid  
> reflection of
> the search time in an actual implementation. Also note that the  
> cataloging
> data is from earlier this summer and may not reflect most recent  
> changes.
> The actual implementation is not going to have all the currently  
> available
> options; the purpose of this evaluation is to find the best  
> algorithm. The
> "start user" needed for some search algorithms will eventually be  
> determined
> automatically from your preferences and profile information. Please,  
> for
> now, just use colleagues teaching the same kind of course.
> Everyone who submits more than 10 evaluations before the end of August
> automatically enters into a raffle for an iPod Shuffle.
> Thanks for your help,
> - Gerd.