CSCI A348/548
Lecture Notes Nine

Fall 2000

Regular expressions in Perl. CGI with POST.

3. Regular expressions

A regular expression is a way of describing a set of strings without having to list all of the strings in the set.

We start from exact patterns, like the string foo, or abc and we introduce quantifiers: * and +.

A character followed by * describes a string of zero or more such characters. Thus

/aba/

refers to the pattern

aba

and

/ab*a/

refers to the pattern that starts with a, is followed by zero or more b's and ends with an a.

* specifies that the preceding character can appear zero or more times. + has a similar meaning, it says that the character appears at least once. * and + are two of a set of characters that have a special meaning and are therefore called metacharacters. They are listed below:

\ | ( [ { ^ $ * ? .

We'll mention two of them, ( and [, and then we'll move on.

( together with its associate ) can be used to capture and memorize the patterns that match. These patterns are being captured in special variables: $1, $2, $3, and so forth. The numbers represent the order of the parens in the pattern.

Example:

$x = "abbbc"; 
$x =~ /a(b*)c/; 
print $1;

will print

bbb

In other words if the pattern specified inside the leaning toothpicks matches, then $1 (which is a special variable) immediately becomes whatever the parens are enclosing.

3.1 Classes of characters

The square bracket is used just as { and }'s are used in mathematics to denote sets, althought the notation is somewhat different.

[a-z] means one alphabetic lowercase character

[a-zA-z] means one alphabetic character

[0-9] means a digit

[a-zA-Z0-9_] is also shortened \w

[0-9] is also shortened \d

[^0-9] means anything but digit

[^\w] is also shortened \W

[ \t\r\n\f] is white space also shortened \s

\n is newline
\r is carriage return
\f is formfeed
\t is tab
there's a blank space ( ) at the beginning

4. Four examples

1. Here's a program that puts parens around a's in the strings that it receives from the command line.

tucotuco.cs.indiana.edu% cat sub
#!/usr/bin/perl
$ARGV[0] =~ s/(a)/($1)/g; 
print $ARGV[0], "\n"; 
tucotuco.cs.indiana.edu% ./sub abcdefghabcdefgh
(a)bcdefgh(a)bcdefgh
tucotuco.cs.indiana.edu% ./sub "abc def gha"
(a)bc def gh(a)

Note the use of double quotes to specify a string with blank spaces in it.

2. Here's another program that does the same thing with any alphabetic character:

tucotuco.cs.indiana.edu% cat sub1
#!/usr/bin/perl
$ARGV[0] =~ s/([a-zA-Z])/($1)/g;
print $ARGV[0], "\n"; 
tucotuco.cs.indiana.edu% ./sub1 "a1 bc3 4_&c +=m "
(a)1 (b)(c)3 4_&(c) +=(m)

3. Here's a program that reads the index.html file and prints out the lines that have what looks like a hyperlink on them:

open (AB, "/u/dgerman/httpd/htdocs/index.html");
while ($x = <AB>) {
  if ($x =~ /<a href="([^"]+)">([^<]+)<\/a>/) {
    print $1; 
  } 
} 
close(AB);

The two patterns in round parens are non-empty strings that will be stored in $1 and $2 after they match. The first one is a string that contains at least one character and does not contain double quotes. (This makes the pattern matching mechanism stop at the first " encountered double quote).

The second one describes a non-empty (+) string of characters that does not contain the < sign (which is where the description of the hyper-reference ends).

If you look close you will see outside these two patterns the clear structure of an

<a href="...">...</a>

tag, except we have put those two intimidating patterns where the ellipsis are.

4. Lines in access_log start like this:

129.79.207.219 - - [16/Sep/1999:01:29:37

This can be described as follows:

^[\S]+ - - \[[^:]+:\d\d:\d\d:\d\d

that is:

^ outside ['s means start of string
followed by a non-empty string of non-blank (\S) chars
followed by the - - pattern
then by a square bracket (protected with \ from acting as a metacharacter)
then by a non-empty string of non-:'s
followed by the time pattern: two digits (for the hour), the : character, two digits (for the number of minutes), again :, and two digits for the number of seconds

This is not the only possible description, just one that suits our purpose; having it we can collect this information to build a table of the number of hits, grouped by hour, for the server.

open (AB, "httpd/logs/access_log");
while ($x = <AB>) {
  if ($x =~ /^([\S]+) - - \[([^:]+:\d\d):\d\d:\d\d/) {
    $hits{$1} += 1; 
  } 
} 
close(AB);

The first pair of parens collects the IP number, the second one a date like this:

16/Sep/01

that means Sept 16, and the time 1am.

For each request to the server there is a line in the log file. Each line has the time of access. We basically count the lines (which stand for hits) and put them in bins, one such bin for each distinct hour of our server's life.

5. CGI.pm

We have developed a ReadParse function last time that does the reading and parsing for us within a CGI script. This function was developed following Steve Brenner's cgi-lib.pl library. This library is mentioned in your book on page 503. We only developed a core part of it, the most basic part of it that does CGI processing.

The function looked like this:

sub ReadParse {
  local ($i, $key, $val) = @_; 

  if ($ENV{'REQUEST_METHOD'} eq 'GET' ) {
    $in = $ENV{'QUERY_STRING'}; 

  } elsif ($ENV{'REQUEST_METHOD'} eq 'POST') { 
    read (STDIN, $in, $ENV{'CONTENT_LENGTH'}); 
  } 

  @in = split(/&/, $in); 

  for ($i = 0; $i <= $#in; $i++) {
    $in[$i] =~ s/\+/ /g; 

    ($key, $val) = split(/=/, $in[$i]); 

    $key =~ s/%(..)/pack("c", hex($1))/ge; 
    $val =~ s/%(..)/pack("c", hex($1))/ge;     

    if (defined($in{$key})) {
      $in{$key} .= "\0"; # thanks to Rudy!
    } 

    $in{$key} .= $val; 
  }   
}

You can use this in all your CGI scripts to process incoming data.

For example here's a circular script:

#!/usr/local/bin/perl
 
&ReadParse; 
 
&header("Lab 5 Circular Script");
 
if      ($ENV{REQUEST_METHOD} eq 'GET' ) {
  &printform; 
} elsif ($ENV{REQUEST_METHOD} eq 'POST') {
  &printform($in{count}); 
} 
 
&trailer; 
 
sub printform {
  local ($arg) = @_; 
  local $count = $arg + 1;
  print qq{
      <form method="POST" action="$ENV{SCRIPT_NAME}"> 
          Your call has number: <font size=+5>$count<font>. <p>  
      Press <input type="submit" value="here"> to call again. 
      <input type="hidden" name="count" value="$count"> 
      </form>
  }; 
} 
 
sub header {
    local ($t) = @_; 
  print "Content-type: text/html\n\n<html><head>";
  print "<title>$t</title></head><body bgcolor=white>\n"; 
} 
 
sub trailer {
  print "\n</body></html>"; 
}
 
sub ReadParse {
  local ($i, $key, $val) = @_; 
 
  if ($ENV{'REQUEST_METHOD'} eq 'GET' ) {
    $in = $ENV{'QUERY_STRING'}; 
 
  } elsif ($ENV{'REQUEST_METHOD'} eq 'POST') { 
    read (STDIN, $in, $ENV{'CONTENT_LENGTH'}); 
  } 
 
  @in = split(/&/, $in); 
 
  for ($i = 0; $i <= $#in; $i++) {
    $in[$i] =~ s/\+/ /g; 
 
    ($key, $val) = split(/=/, $in[$i]); 
 
    $key =~ s/%(..)/pack("c", hex($1))/ge; 
    $val =~ s/%(..)/pack("c", hex($1))/ge;     
 
    if (defined($in{$key})) {
      $in{$key} .= "\0";  
    } 
 
    $in{$key} .= $val; 
  }   
}

Try this script.

But notice that our ReadParse can't handle file uploads and is not as robust as it possibly could and should be. We can save some time by using one of the public domain libraries, such as CGI.pm.

CGI.pm is a Perl library for writing CGI. It handles many of the ugly details of creating HTTP headers, parsing query strings, and maintaining the state of fill-out forms so that you can concentrate on the task at hand. The module is widely used and frequently updated. The description in your book starts on page 494.

Last updated on September 24, 2000, by Adrian German for A348/A548.

`[a-z]`	means one alphabetic lowercase character
`[a-zA-z]`	means one alphabetic character
`[0-9]`	means a digit
`[a-zA-Z0-9_]`	is also shortened `\w`
`[0-9]`	is also shortened `\d`
`[^0-9]`	means anything but digit
`[^\w]`	is also shortened `\W`
`[ \t\r\n\f]`	is white space also shortened `\s`
	`\n` is newline `\r` is carriage return `\f` is formfeed `\t` is tab there's a blank space ( ) at the beginning

CSCI A348/548 Lecture Notes Nine

CSCI A348/548
Lecture Notes Nine