Perl Diver 2.31
Main Environment Variables Perl Default Values Perl Config - Summary Perl Config - Full Installed Modules List Directory uptime Docs

Module Documentation
Details and documentation about a specific module, including version and documentation (if available). Note that while links to perldoc.com and search.cpan.org are provided, the module may be part of a larger distribution. If you reach a File Not Found page on either site, please try the parent module.

WWW::Mechanize

Name WWW::Mechanize
Version 1.97
Located at /usr/share/perl5/vendor_perl
File /usr/share/perl5/vendor_perl/WWW/Mechanize.pm
Is Core No
Search CPAN for this module WWW::Mechanize
Documentation WWW::Mechanize
Module Details WWW::Mechanize

NAME

WWW::Mechanize - Handy web browsing in a Perl object

VERSION

version 1.97

SYNOPSIS

WWW::Mechanize supports performing a sequence of page fetches including following links and submitting forms. Each fetched page is parsed and its links and forms are extracted. A link or a form can be selected, form fields can be filled and the next page can be fetched. Mech also stores a history of the URLs you've visited, which can be queried and revisited.

    use WWW::Mechanize ();
    my $mech = WWW::Mechanize->new();
    $mech->get( $url );
    $mech->follow_link( n => 3 );
    $mech->follow_link( text_regex => qr/download this/i );
    $mech->follow_link( url => 'http://host.com/index.html' );
    $mech->submit_form(
        form_number => 3,
        fields      => {
            username    => 'mungo',
            password    => 'lost-and-alone',
        }
    );
    $mech->submit_form(
        form_name => 'search',
        fields    => { query  => 'pot of gold', },
        button    => 'Search Now'
    );
    # Enable strict form processing to catch typos and non-existant form fields.
    my $strict_mech = WWW::Mechanize->new( strict_forms => 1);
    $strict_mech->get( $url );
    # This method call will die, saving you lots of time looking for the bug.
    $strict_mech->submit_form(
        form_number => 3,
        fields      => {
            usernaem     => 'mungo',           # typo in field name
            password     => 'lost-and-alone',
            extra_field  => 123,               # field does not exist
        }
    );

DESCRIPTION

WWW::Mechanize, or Mech for short, is a Perl module for stateful programmatic web browsing, used for automating interaction with websites.

Features include:

Mech is well suited for use in testing web applications. If you use one of the Test::*, like the Test::HTML::Lint manpage modules, you can check the fetched content and use that as input to a test call.

    use Test::More;
    like( $mech->content(), qr/$expected/, "Got expected content" );

Each page fetch stores its URL in a history stack which you can traverse.

    $mech->back();

If you want finer control over your page fetching, you can use these methods. follow_link and submit_form are just high level wrappers around them.

    $mech->find_link( n => $number );
    $mech->form_number( $number );
    $mech->form_name( $name );
    $mech->field( $name, $value );
    $mech->set_fields( %field_values );
    $mech->set_visible( @criteria );
    $mech->click( $button );

the WWW::Mechanize manpage is a proper subclass of the LWP::UserAgent manpage and you can also use any of the LWP::UserAgent manpage's methods.

    $mech->add_header($name => $value);

Please note that Mech does NOT support JavaScript, you need additional software for that. Please check JavaScript in the WWW::Mechanize::FAQ manpage for more.

IMPORTANT LINKS

CONSTRUCTOR AND STARTUP

new()

Creates and returns a new WWW::Mechanize object, hereafter referred to as the ``agent''.

    my $mech = WWW::Mechanize->new()

The constructor for WWW::Mechanize overrides two of the params to the LWP::UserAgent constructor:

    agent => 'WWW-Mechanize/#.##'
    cookie_jar => {}    # an empty, memory-only HTTP::Cookies object

You can override these overrides by passing params to the constructor, as in:

    my $mech = WWW::Mechanize->new( agent => 'wonderbot 1.01' );

If you want none of the overhead of a cookie jar, or don't want your bot accepting cookies, you have to explicitly disallow it, like so:

    my $mech = WWW::Mechanize->new( cookie_jar => undef );

Here are the params that WWW::Mechanize recognizes. These do not include params that the LWP::UserAgent manpage recognizes.

In addition, WWW::Mechanize also allows you to globally enable strict and verbose mode for form handling, which is done with the HTML::Form manpage.

To support forms, WWW::Mechanize's constructor pushes POST on to the agent's requests_redirectable list (see also the LWP::UserAgent manpage.)

$mech->agent_alias( $alias )

Sets the user agent string to the expanded version from a table of actual user strings. $alias can be one of the following:

then it will be replaced with a more interesting one. For instance,

    $mech->agent_alias( 'Windows IE 6' );

sets your User-Agent to

    Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

The list of valid aliases can be returned from known_agent_aliases(). The current list is:

  • Windows IE 6Windows IE 6
  • Windows MozillaWindows Mozilla
  • Mac SafariMac Safari
  • Mac MozillaMac Mozilla
  • Linux MozillaLinux Mozilla
  • Linux KonquerorLinux Konqueror

known_agent_aliases()

Returns a list of all the agent aliases that Mech knows about.

PAGE-FETCHING METHODS

$mech->get( $uri )

Given a URL/URI, fetches it. Returns an the HTTP::Response manpage object. $uri can be a well-formed URL string, a URI object, or a the WWW::Mechanize::Link manpage object.

The results are stored internally in the agent object, but you don't know that. Just use the accessors listed below. Poking at the internals is deprecated and subject to change in the future.

get() is a well-behaved overloaded version of the method in the LWP::UserAgent manpage. This lets you do things like

    $mech->get( $uri, ':content_file' => $tempfile );

and you can rest assured that the params will get filtered down appropriately.

NOTE: Because :content_file causes the page contents to be stored in a file instead of the response object, some Mech functions that expect it to be there won't work as expected. Use with caution.

$mech->post( $uri, content => $content )

POSTs $content to $uri. Returns an the HTTP::Response manpage object. $uri can be a well-formed URI string, a URI object, or a the WWW::Mechanize::Link manpage object.

$mech->put( $uri, content => $content )

PUTs $content to $uri. Returns an the HTTP::Response manpage object. $uri can be a well-formed URI string, a URI object, or a the WWW::Mechanize::Link manpage object.

$mech->reload()

Acts like the reload button in a browser: repeats the current request. The history (as per the back() method) is not altered.

Returns the the HTTP::Response manpage object from the reload, or undef if there's no current request.

$mech->back()

The equivalent of hitting the ``back'' button in a browser. Returns to the previous page. Won't go back past the first page. (Really, what would it do if it could?)

Returns true if it could go back, or false if not.

$mech->clear_history()

This deletes all the history entries and returns true.

$mech->history_count()

This returns the number of items in the browser history. This number does include the most recently made request.

$mech->history($n)

This returns the nth item in history. The 0th item is the most recent request and response, which would be acted on by methods like find_link()|"$mech->find_link( ... )". The 1th item is the state you'd return to if you called back()|/$mech->back().

The maximum useful value for $n is $mech->history_count - 1. Requests beyond that bound will return undef.

History items are returned as hash references, in the form:

  { req => $http_request, res => $http_response }

STATUS METHODS

$mech->success()

Returns a boolean telling whether the last request was successful. If there hasn't been an operation yet, returns false.

This is a convenience function that wraps $mech->res->is_success.

$mech->uri()

Returns the current URI as a URI object. This object stringifies to the URI itself.

$mech->response() / $mech->res()

Return the current response as an the HTTP::Response manpage object.

Synonym for $mech->response()

$mech->status()

Returns the HTTP status code of the response. This is a 3-digit number like 200 for OK, 404 for not found, and so on.

$mech->ct() / $mech->content_type()

Returns the content type of the response.

$mech->base()

Returns the base URI for the current response

$mech->forms()

When called in a list context, returns a list of the forms found in the last fetched page. In a scalar context, returns a reference to an array with those forms. The forms returned are all the HTML::Form manpage objects.

$mech->current_form()

Returns the current form as an the HTML::Form manpage object.

$mech->links()

When called in a list context, returns a list of the links found in the last fetched page. In a scalar context it returns a reference to an array with those links. Each link is a the WWW::Mechanize::Link manpage object.

$mech->is_html()

Returns true/false on whether our content is HTML, according to the HTTP headers.

$mech->title()

Returns the contents of the <TITLE> tag, as parsed by the HTML::HeadParser manpage. Returns undef if the content is not HTML.

CONTENT-HANDLING METHODS

$mech->content(...)

Returns the content that the mech uses internally for the last page fetched. Ordinarily this is the same as $mech->response()->decoded_content(), but this may differ for HTML documents if update_html is overloaded (in which case the value passed to the base-class implementation of same will be returned), and/or extra named arguments are passed to content():

$mech->content( format => 'text' )
Returns a text-only version of the page, with all HTML markup stripped. This feature requires HTML::TreeBuilder version 5 or higher to be installed, or a fatal error will be thrown. This works only if the contents are HTML.

$mech-content( base_href => [$base_href|undef] ) >>$mech->content( base_href => [$base_href|undef] )
Returns the HTML document, modified to contain a <base href="$base_href"> mark-up in the header. $base_href is $mech->base() if not specified. This is handy to pass the HTML to e.g. the HTML::Display manpage. This works only if the contents are HTML.

$mech-content( raw => 1 ) >>$mech->content( raw => 1 )
Returns $self->response()->content(), i.e. the raw contents from the response.

$mech-content( decoded_by_headers => 1 ) >>$mech->content( decoded_by_headers => 1 )
Returns the content after applying all Content-Encoding headers but with not additional mangling.

$mech-content( charset => $charset ) >>$mech->content( charset => $charset )
Returns $self->response()->decoded_content(charset => $charset) (see the HTTP::Response manpage for details).

To preserve backwards compatibility, additional parameters will be ignored unless none of raw | decoded_by_headers | charset is specified and the text is HTML, in which case an error will be triggered.

$mech->text()

Returns the text of the current HTML content. If the content isn't HTML, $mech will die.

The text is extracted by parsing the content, and then the extracted text is cached, so don't worry about performance of calling this repeatedly.

LINK METHODS

$mech->links()

Lists all the links on the current page. Each link is a WWW::Mechanize::Link object. In list context, returns a list of all links. In scalar context, returns an array reference of all links.

$mech->follow_link(...)

Follows a specified link on the page. You specify the match to be found using the same params that find_link()|"$mech->find_link( ... )" uses.

Here some examples:

Returns the result of the GET method (an the HTTP::Response manpage object) if a link was found.

If the page has no links, or the specified link couldn't be found, returns undef. If autocheck is enabled an exception will be thrown instead.

$mech->find_link( ... )

Finds a link in the currently fetched page. It returns a the WWW::Mechanize::Link manpage object which describes the link. (You'll probably be most interested in the url() property.) If it fails to find a link it returns undef.

You can take the URL part and pass it to the get() method. If that's your plan, you might as well use the follow_link() method directly, since it does the get() for you automatically.

Note that <FRAME SRC="..."> tags are parsed out of the the HTML and treated as links so this method works with them.

You can select which link to find by passing in one or more of these key/value pairs:

If n is not specified, it defaults to 1. Therefore, if you don't specify any params, this method defaults to finding the first link on the page.

Note that you can specify multiple text or URL parameters, which will be ANDed together. For example, to find the first link with text of ``News'' and with ``cnn.com'' in the URL, use:

    $mech->find_link( text => 'News', url_regex => qr/cnn\.com/ );

The return value is a reference to an array containing a the WWW::Mechanize::Link manpage object for every link in $self->content.

The links come from the following: