Sporadic loss of FORM scope data

[kriskadela]

Up to 10% of my users get errors when submitting forms. FORM scope is totally
gone (no FIELDNAMES, no hidden fields, zip, nada)

I have done the same simple (3 hidden form fields and 2 text fields)
application is ASP and don't get any such errors.

What's going on with ColdFusion? I get this same problem on CF 5, 6.1 and now
7 on 2 seperate servers and 2 different service providers. I am beginning to
think that it is time tp ditch CF. This has been going on for ages and I have
yet to have any response from anyone at Macromedia.

[PaulH]

this is the first i've heard of this (and searching the forums only turns up
your recent posts). i guess we'll need to see more info. do you have a simple
example and maybe some indication (what kind of load, etc.) of when this
occurs?

[JMGibson3]

You are not alone. We get it very infrequently (1 or 2 a month) on a
prime-time intranet of about 40,000 pages per month. No pattern at all. This
is back on CF 4.5.1 SP2 and has also been going on for years. When we think of
it (or more often nowadays, inherit it when cloning) we throw in some
IsDefined('Form.xxx') checks and redirect to an error page if they've vanished.
Everything here is IIS and IE. One theory is that button mashers are getting
a page with form fields submitted before IE has finished displaying it. Who
knows.

[kriskadela]

There is no pattern and it happens under light and heavy load. It also happens
(although less frequently percentage wise) with other browsers like Firefox and
Safari. I know that ASP will exhibit this behaviour when a site uses mixed
authentication schemes but this is not the case here (all default IIS and CF
settings - default anon IIS account).

Another very frustrating point is that I simply have not been able to
reproduce this on any of my own workstations no matter how I mess with any of
the browser settings. I have had friends run tests and never got anything to
fail even once.

I don't have an example since this is happening with every application from
simple pages with 4-5 fields to complex ones with over 500 form fields.

Often when the FORM scope disappears the client scope is also lost (I don't
use CFLOCATION anywhere and I do pass the tokens from page to page on the URL).

All servers are relatively new (Dual 2.4GHz Xeon) and have at least 2GB RAM.

[Mike Greider]

Is any other information lost, or is it just the information in the form scope?
Secondly, does the form scope completely disappear, or just specific values
from within the scope? In our 5+ years with CF we've never seen this, but would
be happy to help you troubleshoot.

[JMGibson3]

Don't know for sure if it's Form or everything, or even all Form vs. a field or
2. It's just something that shows up on our Site-Wide Error Handler for those
few pages that remain that don't check for IsDefined and redirect. Our
redirects are page to the origin page with a 'sorry, form variables were lost,
please re-enter'. In most cases, this will refresh from Session variables and
our users have never complained. So I've just realized that the 1 or 2 per
month incident rate is actually (much?) higher but have no clue by how much.
I'm also sure there are some cases (different team members, coding styles)
where the clients DO have to re-key everything. We're Intranet not e-commerce
so we're usually only talking a few fields on a Query Selection page. I have
personally gotten pretty angry myself at sites that lose my entry of dozens of
fields or lengthy text. I've often wondered if Form fields can 'expire'.
Someone keys in a bunch of fields, gets interrupted and clicks Submit upon
their return hours later. 95% of our Forms use Method='Post'. Post vs. Get
has always confused the heck out me. We use 'Post' because that's what the
first person used. No idea if this is appropriate or wise. I DO understand
that Post involves some Client/Web Server back chatter. Can 'Post' forms go
stale and result in expired Form fields?

[Mike Greider]

Can you post some of the pages that have this problem? Maybe another set of
eyes can help identify some issue. Use POST with your forms. A GET will send
all of the form fields and values through the URL bar. Obviously, if you have
anything other than a really small form, you'll exceed the URL length. Do you
have cache prevention on any of the pages? Even so, you should lose any form
fields. Maybe someone is submitting the form because the entire form has been
loaded up? But that's probably a bit of a long shot because I'm assuming that
your submit is at the bottom of the form.

[kim il sung]

intresting.
do you use cfforms or plain html forms?
can some javascript/vbscript etc reload/submit page ?
form field names containing any specials (for cf)?
Kim

[kriskadela]

Well, we use plain HTML forms, POST, with CFID and CFTOKEN passed on the URL
Some of our forms have more than 250 elements (radio buttons)
The whole thing is as simple as it gets

When the form scope is lost so are the client and session scopes. The whole
scope is gone. The CFID/CFTOKEN cookies are present on the browser and in the
client variable database.

Anyone I have ever asked (including nmyself) to test has been unable to
encouter the problem but it is there as I get an email copy of every error and
a dump of each variable scope.

Here, you can have a go at my test account.
[url]http://www.e-selecsys.com/index.cfm?OfficeID=demo11[/url]

There is nothing fancy as far as processing goes. Just simple database updates
and cfinclude of next part with last page using a COM object to crunch the
numbers. In some other applications I submit the page to itself for
verification but that does not decrease or increase the number of errors.

Query string (URL) variables are never lost.

[Mike Greider]

Can you post the code for us to look at? Why have you added the cfide and
cftoken to the url in the form post? Not sure, but this might have something to
do with it. YOu shouldn't need to manually pass these fields unless you are
using a cflocation. The reason is that cflocation is a redirect where the
cookies are passed back to the server. In regular form post, this isn't the
case. Try removing those from the form url and seeing if there is a difference.
On long shot is that their cfide/cftoken is changing and since you've hardcoded
a different cfide/cftoken, the client variables are no longer matching up.
THat's a long shot though. You said that the query string variables are never
lost? What are those? Just the cfide/cftoken variables?

[Kronin555]

The query string is everything after the ? in a URL

In the following URL:
[url]http://www.example.com/foo/bar/page.cfm?val1=jar&val2=ball[/url]

/foo/bar/page.cfm is CGI.SCRIPT_NAME
and
val1=jar&val2=ball is CGI.QUERY_STRING

[Mike Greider]

I'm sorry. I should have been more specific. I understand query strings, I
just wanted to know what was being reported as the query string in kriskadela's
program. In his example, the query string should only have been the
cfid/cftoken values, and i just wanted to confirm.

[kriskadela]

We use client scope variables to track progress from page to page and to store
other bits of temporary data (nothing big, just some boolean stuff).

Since we use client scope variables it is considered to be a good practice to
pass the CFID and CFTOKEN to every page in case cookies get messed up.

As far as the code goes

step 1: format the data <cfset
Survey1="#FORM.S1_1##FORM.S1_2#.....#FORM.S1_X#">
step 2: Save the data <CFQUERY>UPDATE Surveys SET Survey1='#Survey1#' WHERE
......</CFQUERY>
step 3: get next survey <CFINCLUDE TEMPLATE="survey2.inc">

repeat from step 1 for the other parts

Like I said, when the FORM scope is lost so are the client and session scopes.

I don't see how on earth the client scope can just vanish when CFID and
CFTOKEN are passed to every page and they are present in the client variable
storage database (CF is supposed to check cookies first then URL to get the
value of CFID and CFTOKEN if cookies not present).

Just loading this forum generated 2 errors (something to do with Invalid
Session and then exceeding number of concurrent users). Thsi does not give me
any confidence in CF anymore if the company selling it can't make a relaible
site (not to mention that it is slow).

[Mike Greider]

I'm not sure it is good practice to always manually pass the cfid/cftoken
params on every page. ColdFusion does that for you. You only need to manually
pass these params if the cookies will not be sent to the server, i.e., using
cflocation. Otherwise, what if someone bookmarks a page that has old
cfid/cftoken params on it? Is it possible that the users are reloading the
last page, or bookmarking the last page? If they are bookmarking, than they
could come into that page without submitting a form. Perhaps the last page is
always expecting a form submission? If so, that would explain why the
cfid/cftoken params exist, but not the other information.

[JMGibson3]

OK all. I've found the answer for myself at least. (Kris says he loses the
entire Form and sometimes Client scopes which is not my case.) At any rate the
Button-Masher theory proposed by my colleagues here and reiterated by Mike
Greider has been proven true. It most definitely IS POSSIBLE to submit a Form
before it has finished loading and the bottom-most fields will be missing.
I've pasted the Form and Action pages I used to proved the principle at the
bottom. It seems to be browser independent but I only have IE and Firefox to
test. Firefox seems to be a little more resistant in that he waits awhile
before the top of the page starts appearing, IE starts right away. If you
adjust the loop setting in the Form page, eventually you can arrive at a value
where you can hit the refresh button and start mashing the Run Query button as
soon as it appears and the final form field2 will be missing at the action
page, displays and diagnostics follow: Button Arrived: Run Query Field1
Arrived: 1 Form Field2 missing Parameters Form Fields: BUTTON=Run Query
FIELD1=1 FIELDNAMES=BUTTON,FIELD1 PASSWORD= USERNAME= As seen above, however,
the first part of the Form scope is present. I had to get the loop up to 50000
for my own quick Intranet. In our live sporadic cases it is most likely NOT
the amount of content. It's probably traffic congestion, sputtering switches
and routers, wireless, decrepit PC's or a combination. Whatever the reason all
it needs is for the Submit button to appear before all of the Form Fields
appear. Obviously they didn't built it into the ANSI Browser Specifications,
so does anyone know of a Javascript workaround that will disable all buttons
until all Form Fields have arrived?

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>Page_Test_Form</title>
</head>
<body>
<CFSET varActionPage="action_Test_Form.cfm">

<!--- Display form to set query parameters and run it --->
<CFOUTPUT>
<CFFORM ACTION="#varActionPage#" METHOD="POST" name="frmGenQry">
<TABLE width=90% align="center" border="0" rules="rows">
<TR>
<TD width=5%></TD>
<TD align="center" width=20%>
<INPUT TYPE="SUBMIT" NAME="Button" VALUE="Run Query"></TD>
<TD width=50%></TD>
<TD align="center" width=20%>
<INPUT TYPE="SUBMIT" NAME="Button" VALUE="Clear Query"></TD>
<TD width=5%></TD>
</TR>
</table>

<TABLE width=100% align="center" border="4" rules="all">
<TR>
<TD CLASS=DisplayColumnHeadingRight><B>Selections:</B></TD>
<TD CLASS=DisplayColumnHeadingLeft><B>Values:</B></TD>
</TR>
<TR>
<TD CLASS=DisplayDataLabelNormal><B>Field One:</B></TD>
<TD><SELECT NAME="Field1">
<CFLOOP INDEX="LoopCount" FROM="1" TO="5">
<OPTION> #LoopCount#
</CFLOOP>
</SELECT></TD>
</TR>
<tr><td colspan=2>One line</td></tr>
<CFLOOP INDEX="LoopCount" FROM="1" TO="50000">
<tr><td colspan=2>and another #LoopCount#,</td></tr>
</CFLOOP>
<TR>
<TD CLASS=DisplayDataLabelNormal><B>Field 2:</B></TD>
<TD><SELECT NAME="Field2">
<CFLOOP INDEX="LoopCount" FROM="1" TO="5">
<OPTION> #LoopCount#
</CFLOOP>
</SELECT></TD>
</TR>
</TABLE>
</CFFORM>
</CFOUTPUT>

</body>
</html>

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>Action_Test_Form</title>
</head>
<body>

<cfif IsDefined("Form.Button")>
<p>Button Arrived: <cfoutput>#Form.Button#</cfoutput></p>
<cfif IsDefined("Form.Field1")>
<p>Field1 Arrived: <cfoutput>#Form.Field1#</cfoutput></p>
<cfif IsDefined("Form.Field2")>
<p>Field2 Arrived: <cfoutput>#Form.Field2#</cfoutput></p>
<cfelse>
<p>Form Field2 missing</p>
</cfif>
<cfelse>
<p>Form Field1 missing</p>
</cfif>
<cfelse>
<p>Form Button missing</p>
</cfif>

</body>
</html>

[Mike Greider]

I don't know about the JavaScript solution, but if you want to absolutely sure
that this was the problem, try using a proxy tool that will let you inspect
what fields are submitted to the server. I use Achilles and LiveHTTPHeaders
(built right into Firefox). Then, submit the form and inspect what is actually
sent to the server. You can see if form fields are missing. Then you can
completely run out ColdFusion being the problem.

[JMGibson3]

Mike, I installed LiveHTTPHeader and ran the test in FireFox with it turned
on in the side bar. Following results appeared. Am I reading this right, that
it proves that the Browser never sent field2 rather than IIS (or especially
CF) dropped it? (I never really thought it was a CF problem in the first
place) [url]http://obfuscated/action_Test_Form.cfm[/url] POST
/obfuscated/action_Test_Form.cfm HTTP/1.1 Host: wbkapps.na.sappi.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5)
Gecko/20041107 Firefox/1.0 Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,
image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding:
gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300
Connection: keep-alive Referer: [url]http://obfuscated/page_Test_Form.cfm[/url] Cookie:
CFTOKEN=66240279; CFID=496313 Content-Type: application/x-www-form-urlencoded
Content-Length: 25 Button=Run+Query&amp;Field1=1 HTTP/1.x 200 OK Server:
Microsoft-IIS/4.0 Date: Thu, 07 Apr 2005 20:46:14 GMT Connection: close
Content-Type: text/html Page-Completion-Status: Normal, Normal
----------------------------------------------------------

[Mike Greider]

Yep. You are reading it right. It's not ColdFusion's fault.

Wow ... not too many people on these forums will say that. :D

[kim il sung]

simple javascript solution:
last visible form field contains js to enable submit button to work via hidden
form field and
double check with onsubmit function to check if hidden form field contains
boolean info to verify
submit button state
Kim

[kriskadela]

The problem never happens with ASP (3 million applicants per day!) so CF has to
be a factor (1-2 thousand from the 3 million that are redirected to the CF
application and that is where the problems start) and just can't see how
someone can be that fast filling in 50 questions as to submit a form before it
loads. Having people reenter that data is not a solution when they have already
spent 90 minutes entering it in the first place.