Professional Web Applications Themes

Delayed WEB Page Response - PERL Modules

I wish to capture data from a Web page e.g. "http://www.eppraisal.com/PropertyInfo.aspx?a=1215%20Jefferson %20Ave&z=46201" I am using the LWP modules. The page responds in three steps and I have succeeded in capturing only the first. The page first paints up nicely with "Loading" text in the area of interest. After a delay the "Loading" text is replaced with "Calculating". Shortly thereafter, sometimes apparently instantaniously, the data of interest appears. I have tried LWP:: UserAgent and LWP::Parallel::UserAgent and capture only the initial response. TimeOut parameters do not change the behavior. The callback subroutine indicates the HTML comes in several chunks. How can the ...

  1. #1

    Default Delayed WEB Page Response

    I wish to capture data from a Web page e.g.
    "http://www.eppraisal.com/PropertyInfo.aspx?a=1215%20Jefferson
    %20Ave&z=46201"

    I am using the LWP modules.
    The page responds in three steps and I have succeeded in capturing
    only the first.

    The page first paints up nicely with "Loading" text in the area of
    interest.
    After a delay the "Loading" text is replaced with "Calculating".
    Shortly thereafter, sometimes apparently instantaniously, the data of
    interest appears.

    I have tried LWP:: UserAgent and LWP::Parallel::UserAgent and capture
    only the initial response.
    TimeOut parameters do not change the behavior.
    The callback subroutine indicates the HTML comes in several chunks.

    How can the other responses be captured?
    The doentation mentions LPW::Parallel::UserAgent::Entry objects
    and follow up requests.
    Will this be of help?
    I have found no doentation of this feature.
    Is there any additional doentation or examples?

    Thanks,

    AAge Gribskov
    aage.gribskov@gmail.com Guest

  2. #2

    Default Re: Delayed WEB Page Response

    com wrote: 

    It's using javascript - which neither LWP nor WWW::Mechanize will
    execute - to move between pages. You could try using
    Win32::IE::Mechanize or Selenium, but both of these rely on controlling
    a running browser.

    Mark
    Mark Guest

  3. #3

    Default Re: Delayed WEB Page Response

    On Feb 7, 5:00pm, Mark Clements <fr>
    wrote: 




    >
    > It's using javascript - which neither LWP nor WWW::Mechanize will
    > execute - to move between pages. You could try using
    > Win32::IE::Mechanize or Selenium, but both of these rely on controlling
    > a running browser.
    >
    > Mark[/ref]

    There is an API to some of our data. What data elements are you
    looking to pull?

    Send me an email or to info (at) eppraisal.com. Scraping the front-end
    is time consuming and prone to errors (when we push out updates).

    Damian (from eppraisal.com)
    damians@eppraisal.com Guest

Similar Threads

  1. Question Code to save a page getting after response.redirect
    By Sunaina in forum Brainstorming Area
    Replies: 0
    Last Post: January 18th, 05:18 AM
  2. Replies: 0
    Last Post: December 19th, 03:05 PM
  3. CF 7.0 Delayed Response?
    By michaelmuller in forum Coldfusion Server Administration
    Replies: 1
    Last Post: September 9th, 08:54 PM
  4. Replies: 8
    Last Post: September 24th, 12:09 AM
  5. Response Form in Publisher Web Page
    By David Bartosik - MS MVP in forum Web Design
    Replies: 0
    Last Post: August 25th, 03:21 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139