Subject: [LON-CAPA-dev] Re: [LON-CAPA-cvs] cvs: loncom /interface londocs.pm

From: Guy Albertelli II <[email-masked]>
Date: Fri, 7 Feb 2003 13:10:35 -0500 (EST)

Hi All,

I want to use this commit to reinforce a lesson in programming in for
the web.

If you are building a URL, and that URL contains user provided
information _Please_ use 

&Apache::lonnet::escape()

on the user provided data, it will make sure strange characters are
safely passed around.

     if ($isfolder) {
             $url.='&foldername='.
                 &Apache::lonnet::escape($foldertitle);
     }
     $line.='<td bgcolor="#FFFFBB"><a href="'.$url.'" target="cat_'.$folder.
            '"><img src="/adm/lonIcons/'.
            $icon.'.gif" border="0"></a></td>'.

I would urge you that whenever there is a filed that you can type in
that you put in spaces, $, %, /, :  at a minnimum. These all are rather
bad characters that can break one or anther part of the code.

-- 
[email-masked]          BM: n^20 t20 z20 qS 
Guy Albertelli -7-7-4-  O-
    Having control over myself is nearly as good as having control
    over others.

---

From: "H. K. Ng" <[email-masked]>
Date: Fri, 07 Feb 2003 13:31:31 -0500

If you have a module that shows how to extract the problems from a page or 
sequence, please let me know. I like to do the manual grading per 
page/sequence for a given student.

Thanks,
HK

---

From: Jeremy Bowers <[email-masked]>
Date: Fri, 07 Feb 2003 16:41:37 -0500

H. K. Ng wrote:
> If you have a module that shows how to extract the problems from a page 
> or sequence, please let me know. I like to do the manual grading per 

One of the side effects of the recent changes to the nav maps is a 
(hopefully) nice interface to that sort of thing.

I'd start with "perldoc lonnavmaps.pm" (I don't think it's installed 
into the man pages, which I should probably fix), and let me know if you 
have any questions.

The basic routine goes like this:

1. Get a nav map object, which usually looks like this:

     my $navmap = Apache::lonnavmaps::navmap->new(
                         $ENV{"request.course.fn"}.".db",
                         $ENV{"request.course.fn"}."_parms.db", 1, 1);

    because the %ENV has that stuff for the current course already.

2. Somehow, figure out the Resource ID of the map you're interested in.

3. Get the resource object for that map:

     my $mapId = "6.1"; # or whatever
     my $map = $navmap->getById($mapId);

4. The $map has two methods of interest, map_start and map_finish, which
    are the first and last resources in the map. Now you want to
    construct an iterator, telling it to start and stop on those
    resources:

     my $iterator = $navmap->getIterator($map->map_start(),
         $map->map_finish())

5. Now you can query the iterator for the resources, in order. The while
    loop typically looks like this, as you can see in various places in
    the navmaps:

     my $depth = 1;
     $iterator->next(); # skip the first BEGIN_MAP
     my $curRes = $iterator->next(); # for "current resource"

     while ($depth > 0) {
         if($curRes == $iterator->BEGIN_MAP) { $depth++; }
         if($curRes == $iterator->END_MAP) { $depth++; }

         if(ref($curRes)) {
             # your code here, where $curRes is a resource object as
             # documented in lonnavmaps; there's a whole lotta stuff it
             # does
         }

         $curRes = $iterator->next(); # don't forget this or you
                                      # infinitely loop
     }

This will present each resource in order inside of the if(ref($curRes)) 
area. As I've shown here, it will *not* descend into any maps that may 
be contained in the map you are investigating. To change that, make the 
iterator call

     my $iterator = $navmap->getIterator($map->map_start(),
         $map->map_finish(), undef, 1);

which will recurse into all maps under the current one. (In that case 
you'll want to keep track of the depth.)

All the code lined up in a row:

     my $navmap = Apache::lonnavmaps::navmap->new(
                         $ENV{"request.course.fn"}.".db",
                         $ENV{"request.course.fn"}."_parms.db", 1, 1);

     my $mapId = "6.1"; # or whatever
     my $map = $navmap->getById($mapId);

     my $iterator = $navmap->getIterator($map->map_start(),
         $map->map_finish())

     # or
     # my $iterator = $navmap->getIterator($map->map_start(),
     #     $map->map_finish(), undef, 1);

     my $depth = 1;
     $iterator->next(); # skip the first BEGIN_MAP
     my $curRes = $iterator->next(); # for "current resource"

     while ($depth > 0) {
         if($curRes == $iterator->BEGIN_MAP) { $depth++; }
         if($curRes == $iterator->END_MAP) { $depth++; }

         if(ref($curRes)) {
             # your code here, where $curRes is a resource object as
             # documented in lonnavmaps; there's a whole lotta stuff it
             # does
         }

         $curRes = $iterator->next(); # don't forget this or you
                                      # infinitely loop
     }

More information about how to control how this works (such as 
selectively recursing into certain maps) is available on the lonnavmaps 
page. Please let me know if you have any problems; if the documentation 
is incomprehensible or inaccurate on some point, that's a bug.

If your sequence may contain things other then problems, you can use 
$curRes->is_problem() to check whether something is a problem.

(One of the things I'm working on right now is a filtering ability, so 
you can ask the iterator for just problems directly, because I need it 
elsewhere, but it isn't written quite yet so it certainly won't be 
available until the next release at the soonest. ;-) )

---

From: "H. K. Ng" <[email-masked]>
Date: Wed, 19 Feb 2003 15:38:54 -0500

At 04:41 PM 2/7/03 -0500, you wrote:
>I'd start with "perldoc lonnavmaps.pm" (I don't think it's installed into 
>the man pages, which I should probably fix), and let me know if you have 
>any questions.
>

Presumably you are referring to the ID that is embedded in the symbolic 
name. I was able to get the page from which the problem comes from but 
could not get the symbolic names for other problems from the same page.

Does lonnavmaps has a function to get the answers or submissions without 
any html format?

---

From: Guy Albertelli II <[email-masked]>
Date: Wed, 19 Feb 2003 16:54:15 -0500 (EST)

> name.

I believe he is refereing to the 'bghash' id numbers, take a look
at /adm/test and the bighash dump.

> Does lonnavmaps has a function to get the answers or submissions without 

No.

In order to get the answers without formatting you will need to do the
&Apaceh::lonnet::ssi I showed you in the last email. I have verified
that it works with numerical/string/formula/option response, I will
have rank/match/radiobutton working soon.

To get the student's current submission you will need to use
&Apache::lonnet::restore() like you did elsewhere in grades.pm

-- 
[email-masked]  LON-CAPA Developer  0-7-7-4-

---

From: Jerf <[email-masked]>
Date: Wed, 19 Feb 2003 17:19:54 -0500

Guy Albertelli II wrote:
|>>2. Somehow, figure out the Resource ID of the map you're interested in.
|>
|>
|>Presumably you are referring to the ID that is embedded in the symbolic
|>name.
|
|
| I believe he is refereing to the 'bghash' id numbers, take a look
| at /adm/test and the bighash dump.

I am indeed referring to the 'bighash' numbers, but I *think* that they
are both the same thing, so extracting the ID from the symbolic name
might work too.

Don't quote me on this because I'm not an expert on that symb stuff, but
I think that's how I recall it working while I was poking around in
bighash to write the nav stuff.

---

From: Jerf <[email-masked]>
Date: Thu, 20 Feb 2003 11:48:49 -0500

I wrote:
| I am indeed referring to the 'bighash' numbers, but I *think* that they
| are both the same thing, so extracting the ID from the symbolic name
| might work too.

Nope, I was wrong. They aren't the same, as I just found out in my own work.

If you have the source of the map, and the bighash tied into %hash, you
want something like this:

$map_id = $hash{'ids_' . &Apache::lonnet::clutter($mapSrc)};
my $map = $navmap->getById($map_id);

After that, $map->map_start() and $map->map_finish() will give the start
and end resources, which can be passed to the iterator, as described
before. I'm doing something similar right now so I'm reasonably certain
this will work. ;-)

---

From: Jeremy Bowers <[email-masked]>
Date: Fri, 21 Feb 2003 15:24:04 -0500

> could not get the symbolic names for other problems from the same page.

I just committed a new lonnavmaps that includes a "getResourceByUrl" 
method on the lonnavmaps object, so you can ask the $navmap for the map 
directly if you know the URL. (It turns out to be useful in many 
contexts, not surprisingly.)

Of course if you want this before the next release that doesn't help 
much, but you can look at the code as an example if you want, as the 
"getResourceByUrl" code should work in the .6.2 release, if you want to 
implement it somewhere else or include it in your local copy.

Here's how it's coded (and documented) in the latest lonnavmaps, package 
Apache::lonnavmaps::navmap :

=pod

=item * B<getResourceByUrl>(url): Retrieves a resource object by URL of 
the resource. If passed a resource object, it will simply return it, so 
it is safe to use this method in code like "$res = 
$navmap->getResourceByUrl($res)", if you're not sure if $res is already 
an object, or just a URL. If the resource appears multiple times in the 
course, only the first instance will be returned. As a result, this is 
probably useful only for maps.

=cut

sub getResourceByUrl {
     my $self = shift;
     my $resUrl = shift;

     if (ref($resUrl)) { return $resUrl; }

     $resUrl = &Apache::lonnet::clutter($resUrl);
     my $resId = $self->{NAV_HASH}->{'ids_' . $resUrl};
     if ($resId =~ /,/) {
         $resId = (split (/,/, $resId))[0];
     }
     if (!$resId) { return ''; }
     return $self->getById($resId);
}

---

From: "H. K. Ng" <[email-masked]>
Date: Wed, 26 Feb 2003 13:37:06 -0500

Here is what I want to do.

     my $navmap = Apache::lonnavmaps::navmap-> new(
                                       $ENV{'request.course.fn'}.'.db',
                                       $ENV{'request.course.fn'}.'_parms.db',1, 
1);

# Below is the sym name of the page I am interested in. How do I extract 
the mapId from the sym? I had tried a few things but none of them works.

/fsu/ng/2053c_winter2003/CollegePhysicsA_Spring_2003.sequence___8___fsu/ng/2053c_winter2003/Assignment01.page

                               $map->map_finish());

     my $depth = 1;

     while ($depth > 0) {

         if (ref($curRes) && $curRes->is_problem()) {
# What I want to do here is
(1) to get the sym name for each problem that is enclosed in the above 
sequence/page,
(2) the title of the problem, and
(3) the parts in order.
# Once I have the 3 pieces of info, I can get the other things I need.

# I got it to work extracting the title for each sequence/page in a course 
but not the problems within a sequence/page.

             urRes->compTitle();

        }
         $curRes = $iterator->next();
     }

At 03:24 PM 2/21/03 -0500, you wrote:
>directly if you know the URL. (It turns out to be useful in many contexts, 
>not surprisingly.)
>
>Of course if you want this before the next release that doesn't help much, 
>but you can look at the code as an example if you want, as the 
>the resource. If passed a resource object, it will simply return it, so it 
>is safe to use this method in code like "$res = 
>$navmap->getResourceByUrl($res)", if you're not sure if $res is already an 
>object, or just a URL. If the resource appears multiple times in the 
>_______________________________________________
>LON-CAPA-dev mailing list
>[email-masked]
>http://mail.lon-capa.org/mailman/listinfo/lon-capa-dev

---

From: Jerf <[email-masked]>
Date: Wed, 26 Feb 2003 15:32:02 -0500

H. K. Ng wrote:

First, go ahead and add the "getResourceByUrl" I wrote to
Apache::lonnavsmaps::navmap, so I can use it. I'll leave in the quote
marks so you can see what I changed. Also, I assume the symb of the map
~ is in $mapSymb; correct as necessary.

Note that the parts array is sorted lexigraphically, because the order
information is not preserved by the system. Generally, that gives the
correct order, but it may differ on occasion.

| Here is what I want to do.
|
|     my $navmap = Apache::lonnavmaps::navmap-> new(
|                                       $ENV{'request.course.fn'}.'.db',
|
| $ENV{'request.course.fn'}.'_parms.db',1, 1);
|
| # Below is the sym name of the page I am interested in. How do I extract
| the mapId from the sym? I had tried a few things but none of them works.
|
|
/fsu/ng/2053c_winter2003/CollegePhysicsA_Spring_2003.sequence___8___fsu/ng/2053c_winter2003/Assignment01.page

|
|

~      my ($mapUrl, $id, $resourceUrl) = split(/___/, $mapSymb);
~      my $map = $navmap->getResourceByUrl($mapUrl); # add to navmaps

|
|     my $iterator = $navmap->getIterator($map->map_start(),
|                               $map->map_finish());
|
|
|     my $depth = 1;
|     $iterator->next(); # skip the first BEGIN_MAP
|     my $curRes = $iterator->next(); # for "current resource"
|
|     while ($depth > 0) {
|         if($curRes == $iterator->BEGIN_MAP) { $depth++; }
|         if($curRes == $iterator->END_MAP) { $depth++; }
|
|         if (ref($curRes) && $curRes->is_problem()) {
| # What I want to do here is
| #(1) to get the sym name for each problem that is enclosed in the above
| #sequence/page,

~              my $symb = $curRes->symb();

| #(2) the title of the problem, and

~              my $title = $curRes->compTitle();

| #(3) the parts in order.

~              my $parts = $curRes->parts();

| # I got it to work extracting the title for each sequence/page in a
| # course but not the problems within a sequence/page.
|
|        }
|         $curRes = $iterator->next();
|     }
|
|
|
|
|
| At 03:24 PM 2/21/03 -0500, you wrote:
|
|> H. K. Ng wrote:
|>
|>> Presumably you are referring to the ID that is embedded in the
|>> symbolic name. I was able to get the page from which the problem
|>> comes from but could not get the symbolic names for other problems
|>> from the same page.
|>
|>
|> I just committed a new lonnavmaps that includes a "getResourceByUrl"
|> method on the lonnavmaps object, so you can ask the $navmap for the
|> map directly if you know the URL. (It turns out to be useful in many
|> contexts, not surprisingly.)
|>
|> Of course if you want this before the next release that doesn't help
|> much, but you can look at the code as an example if you want, as the
|> "getResourceByUrl" code should work in the .6.2 release, if you want
|> to implement it somewhere else or include it in your local copy.
|>
|> Here's how it's coded (and documented) in the latest lonnavmaps,
|> package Apache::lonnavmaps::navmap :
|>
|> =pod
|>
|> =item * B<getResourceByUrl>(url): Retrieves a resource object by URL
|> of the resource. If passed a resource object, it will simply return
|> it, so it is safe to use this method in code like "$res =
|> $navmap->getResourceByUrl($res)", if you're not sure if $res is
|> already an object, or just a URL. If the resource appears multiple
|> times in the course, only the first instance will be returned. As a
|> result, this is probably useful only for maps.
|>
|> =cut
|>
|> sub getResourceByUrl {
|>     my $self = shift;
|>     my $resUrl = shift;
|>
|>     if (ref($resUrl)) { return $resUrl; }
|>
|>     $resUrl = &Apache::lonnet::clutter($resUrl);
|>     my $resId = $self->{NAV_HASH}->{'ids_' . $resUrl};
|>     if ($resId =~ /,/) {
|>         $resId = (split (/,/, $resId))[0];
|>     }
|>     if (!$resId) { return ''; }
|>     return $self->getById($resId);
|> }
|>
|> _______________________________________________
|> LON-CAPA-dev mailing list
|> [email-masked]
|> http://mail.lon-capa.org/mailman/listinfo/lon-capa-dev
|
|
|
| _______________________________________________
| LON-CAPA-dev mailing list
| [email-masked]
| http://mail.lon-capa.org/mailman/listinfo/lon-capa-dev
|

---

From: "H. K. Ng" <[email-masked]>
Date: Thu, 27 Feb 2003 09:43:58 -0500

|     while ($depth > 0) {

I made some progress in the sense that I can now extract all the problems 
in a page. However, the symb name as called from $curRes->symb() has all 
the special characters returned as hex as given in the example below.

problemname=fsu%2fng%2f2053c%2dwinter2003%2fAssignment01%2epage___4___fsu%2ffsuphysicslib%2f2053c_w03%2fset1%2fmatchUnits%2eproblem
Title=Turbo-Prop Plane

Question: Is there is function to convert it back?

Thanks, - hk

---

From: "H. K. Ng" <[email-masked]>
Date: Thu, 27 Feb 2003 10:07:46 -0500

Never mind - I figure out why.

---

From: Guy Albertelli II <[email-masked]>
Date: Thu, 27 Feb 2003 13:12:12 -0500 (EST)

Hi Hon-Kie,

> Question: Is there is function to convert it back?

Yes, &Apache::lonnet::unescape()

BTW, this type of escaping is HTTP style escaping.

---

From: "H. K. Ng" <[email-masked]>
Date: Thu, 27 Feb 2003 14:42:22 -0500

Two problems.

1. 	    my $parts = $curRes->parts();

does not return anything?

2. I used the function below but it returns a null string. 
(&Apache::loncommon::get_student_answers($symb,$uname,$udom,$ENV{'request.course.id'} 
gave the right answer so I know the arguments are ok.)

my $subresult=&Apache::lonnet::ssi($request->uri,
				   ('grade_target' => 'analyze'),
				   ('grade_domain' => $domain),
				   ('grade_user' => $user),
				   ('grade_symb' => $symb),
				   ('grade_courseid' => $courseid));
(undef,$subresult)=split(/_HASH_REF__/,$subresult,2);
my %analyze=&Apache::lonnet::str2hash($subresult);

And %analyze is a hash containg the structure of the problem.

Looking at lonnet,

sub ssi {

     my ($fn,%form)=@_;

     my $ua=new LWP::UserAgent;

     my $request;

     if (%form) {
       $request=new HTTP::Request('POST',"http://".$ENV{'HTTP_HOST'}.$fn);
       $request->content(join('&',map { &escape($_).'='.&escape($form{$_}) 
} keys %form));
     } else {
       $request=new HTTP::Request('GET',"http://".$ENV{'HTTP_HOST'}.$fn);
     }

     $request->header(Cookie => $ENV{'HTTP_COOKIE'});
     my $response=$ua->request($request);

     return $response->content;
}

Is this the latest version. I just updated lonnet this morning.

Thanks, hk

---

From: Jerf <[email-masked]>
Date: Thu, 27 Feb 2003 10:13:49 -0500

H. K. Ng wrote:
| Two problems.
|
| 1.         my $parts = $curRes->parts();
|
| does not return anything?

Returns a reference to a list containing the parts.

On non-problems, it returns a reference to an empty list.

I am not aware of any problems with this, so please let me know if you
are still getting "nothing" back from ->parts(). (In perl, "nothing" is
an ambiguous term. "undef"? Empty list? etc.)

---

From: Guy Albertelli II <[email-masked]>
Date: Thu, 27 Feb 2003 15:41:51 -0500 (EST)

Hi Hon-Kie,

> gave the right answer so I know the arguments are ok.)

Can you send me the code youare running so I can test it out here?

I can't think of any reasons why it wouldn't work.

> Is this the latest version. I just updated lonnet this morning.

Yep.

---

From: "H. K. Ng" <[email-masked]>
Date: Thu, 27 Feb 2003 15:23:48 -0500

At 10:13 AM 2/27/03 -0500, you wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>H. K. Ng wrote:
>| Two problems.
>|

I used
           for my $part (@{$parts}) {
             $result.='part='.$part.':<br>';
           }
and then print $result. Nothing is printed which implies that the list 
@{$parts} is empty.
It has to be as problem since it is tested with $curRes->is_problem().

---

From: Guy Albertelli II <[email-masked]>
Date: Thu, 27 Feb 2003 17:43:56 -0500 (EST)

Hi Hon-Kie,

> my %analyze=&Apache::lonnet::str2hash($subresult);

Sorry, the first argument needs to be the URL you are requesting. By
passing in $request->uri it was asking /adm/grades

I commit a change so it will ask the problem instead.

---

From: "H. K. Ng" <[email-masked]>
Date: Thu, 27 Feb 2003 16:07:38 -0500

>Can you send me the code youare running so I can test it out here?

I just did a cvs commit for grades.pm

---

From: "Matthew Brian Hall" <[email-masked]>
Date: Fri, 28 Feb 2003 09:56:48 -0500 (EST)

I can confirm this problem.  I did not have it yesterday but I had not
updated my machine in a few days due to my code being borken.  I am
definately seeing it today.

--
------------------------------------------------------------------
Matthew Hall           LON-CAPA developer         [email-masked]
123 North Kedzie Hall                    Michigan State University
------------------------------------------------------------------

---

From: "H. K. Ng" <[email-masked]>
Date: Fri, 28 Feb 2003 09:57:00 -0500

Hi Guy,

Any ideas why

my $parts = $curRes->parts();

- hk

---

From: "Matthew Brian Hall" <[email-masked]>
Date: Fri, 28 Feb 2003 10:07:27 -0500 (EST)

I just fixed this - a minor log error in lonnavmaps.pm.  cvs update should
take care of it.

---

From: "Matthew Brian Hall" <[email-masked]>
Date: Fri, 28 Feb 2003 10:09:39 -0500 (EST)

> I just fixed this - a minor log error in lonnavmaps.pm.  cvs update should
                              ^^^
"log" is short for "logic" (today only).

Matthew