Queue Cfhttp Request

[tml65]

Hi,

I'm dynamically retrieving data, for one of my (public) web page, via a
cfhttp/get request.

The issue is that the data provider has a limit of one request per second.
There's no limiting algo on their end, i'll just get a "penalty" notice.

So... is there anything I can do to limit/queue my requests to one per second?

Thank You

[ianwinter]

You could create a time loop which gets the current time and increments each
second comparing the current time to the know next time. You could create a
scheduled job, though, that's going to be a hefty load hitting the server. HTH

[tml65]

Hi,

Your first suggestion looks interesting. Would you store the "last accessed/next accessible" value in an Application variable?

Thanks Ian

[ianwinter]

Yeah, you could store it anywhere really, application, session, request depends on how the application is used really.

HTH

[tml65]

Looks like it'll be application then. This is mainly to deal with bots, like
googlebot, indexing my site with a surge of requests. Otherwise, with "human"
traffic, it's not going to be likely I get two page requests within one second.

Thanks again Ian,

Tuan

[Mr Black]

Simplest solution (not necessarily the best one) is something like this:

CFLOCK SCOPE='SERVER'
SLEEP 1 sec
CFHTTP

[tml65]

Hi Mr. Black,

I just want to confirm, your suggestion would effective stop cfml processing on the server (for a second), and then resumes the cfhttp?

Thanks

[Mr Black]

No all other pages/code will run. It will just prevent other pages to issue
CFHTTP. Perhaps a better approach will be: CFLOCK name='somename' timeout=n
SLEEP 1 sec CFHTTP /CFLOCK Where 'n' is the number of CF threads multiplied by
the time needed to process CFHTTP + 1 sec. You can just use a large enough
number here. So, if you access the URL only using CFLOCK with this 'somename',
it will guarantee that HTTP requests are not issued more often that 1 req per
sec. You also need to implement SLEEP for 1 sec. There are some tags/tools
that do this better that just use CFLOOP (CPU wasting time approach). Like
this one: [url]http://www.cftagstore.com/tags/cfxexec.cfm[/url]

[tml65]

Thanks for the clear run down, I think i got it.

The crucial thing here is the "name" attribute, reading the documentation it
looks like this is the exact solution i'm looking for:

"Permits synchronizing access to resources from different parts of an
application."

Thanks again,

Tuan