[LON-CAPA-users] Fwd: Support research and win an iPod Shuffle

[Gerd Kortemeyer]

A message from Lars Hoeppner:

Hi,

I'd like to thank the people who already participated in the evaluation.

However, we still need your input!

To make your participation in the evaluation a little easier, I
propose this procedure:

1) choose a search query (and a start user if necessary, i.e. if
you're not a course instructor in some course)
2) use the keyword-search (algorithm 5) and then give a rating based
on those questions:
- are most of the resources found relevant to you or is a majority not
interesting?
- how long would it take you to select the best ones out of the list?
3) issue the same search query but use a filter algorithm (1-4) (keep
the default options) and give a rating based on:
- compared to the results you found with the step 2 before, do you
feel that all relevant resources are still in the result list?
- are the less relevant resources no longer in the list?
- will it take you less time to select the best resources than with
the complete list from step 2?

This usually shouldn't take you longer than a few minutes and would
already help a great deal.

Ideally, do step 3 for all four filter algorithms. If you want to go
wild, you could also try the extend-links on the result page or even
try changing the parameters.
Note regarding browsers: I recently realized that the search result
pages are "optimized" for Mozilla Firefox. The extend-links somehow
get truncated when using Internet Explorer, so in case you use IE you
can only evaluate the filter algorithms.

Thanks for your help!

-Lars

On Thu, Aug 21, 2008 at 7:55 PM, Gerd Kortemeyer <korte@lite.msu.edu>  
wrote:
> Hi,
>
> Lars Hoeppner, whom some of you met at the LON-CAPA Conference this  
> year, is
> working on his master's degree with the Fern-Universitaet Hagen in  
> Germany.
> His topic is community-supported search in an educational content  
> pool with
> sparse metadata, aka LON-CAPA.
>
> There are a number of possible algorithms for community-supported  
> search,
> and the last phase of his project is evaluation by users.
>
> As you are likely setting up your courses this week, please give it  
> a whirl.
>
> Login to
>
> http://s12.lite.msu.edu/
>
> using your normal usernames and password, and by hand changing the  
> "Domain"
> field to your domain. Then go to the course you want to be working  
> on and
> use Search like you normally would.
>
> You get a choice of mechanisms and parameters. Try out a few and  
> please
> evaluate the search results using the
>
> How well do the results meet your expectations?
> (bad) 1 2 3 4 5 (good)
>
> field at the top. Please don't forget to "Send".
>
> Note that the time it takes for these searches is not a valid  
> reflection of
> the search time in an actual implementation. Also note that the  
> cataloging
> data is from earlier this summer and may not reflect most recent  
> changes.
> The actual implementation is not going to have all the currently  
> available
> options; the purpose of this evaluation is to find the best  
> algorithm. The
> "start user" needed for some search algorithms will eventually be  
> determined
> automatically from your preferences and profile information. Please,  
> for
> now, just use colleagues teaching the same kind of course.
>
> Everyone who submits more than 10 evaluations before the end of August
> automatically enters into a raffle for an iPod Shuffle.
>
> Thanks for your help,
>
> - Gerd.
>
>
>