CFPOP Encoding Problem

[Spiralhead]

So in other words, dont use ColdFusion for building webmail apps? It seems to
me that's what you're saying.

I just finished developing an email based helpdesk-app. the server it's on is
currently running coldfusion 5. How often should I expext these error messages?
I haven't encountered anything like that in testing but we only used a couple
different email clients.

[ksmith]

This issue can be coded around in cfml. Cfpop/getheaders works even for utf-7
and other unsupported charsets. So you can do your normal cfpop/getall in a
cftry/cfcatch block. If you get the unsupported charset exception you can do a
cfpop/getheaders in the catch block. Then loop the headers searching for
utf-7. When you find a bad email you can -reply with a message to the sender
that their email was un-readable and ask that they call, email from a different
client, change their charset, etc. -delete the bad email - continue thru to
find any other bad emails. Finally do another cfpop/getall on the cleaned
server. Use of UTF-7 for email seems very rare and you should not run into
this frequently . I will attach a demo template showing this, shortly.

[RavenCecil]

Ken:

Your method doesn't scale very well for those of us that access thousands of e-mails per day. :(

Any other thoughts?


Warm Regards,

Raven

[dsbrady_98]

The way I found around this (regarding using cfpop to retrieve e-mails) was to
buy the CFX custom tag, CFX_POP3. It was about $20 US ( 10 British pounds )
and seems to be well worth it, if you're able to install a CFX on your server.

[ksmith]

Raven, I am surprised that it does not scale well, since it should only do a
cfpop/getall 99.9% of the time. It should only do the catch block
occassionally. Are you seeing a lot of messages with utf-7 encoding?

[Spiralhead]

I think what he means is that it's a lot of CRAP--polluting what would otherwise be an elegant block of code.

[TLeidinger]

This workaround also doesn't work for me.

Do I expect too much, if I'm saying that an application worth thousands of
Dollars should be able to receive emails???? The developers of CFX_POP3 are
also able to resolve this problem...

[Spiralhead]

I've run into yet another error with the CFPOP tag. I'm posting to this thread
because it seems to be the only way to get a response. Anyway the error
happened with an email message with a 6MB mp3 as an attachment. The error read
as follows:

Error Diagnostic Information

Unknown exception condition

Unknown error while executing tag

It seems to happen when attempting to download the attachment with the CFPOP
tag. I tried enclosing it in a cftry/cfcatch statement but the error persisted.
This is a huge problem and is threatening to derail my entire project. We're
running coldfusion 5 on our web server. Is this a problem that's been addressed
with MX? Please respond. Thanks.

[RavenCecil]

Ken: Sorry, it's not your code, just the mailbox I'm processing. It has to
download/sort about 40000+ emails every day. Sometimes it's impossible to even
connect using POP. I found a great piece of freeware that uses MySQL/MSSQL as
the backend, so now I can bypass the POP aspect and just sort them directly!
Thanks for responding! Warm Regards, Raven

[RavenCecil]

TLeidinger: Yes, especially when the application in question is a RAD tool
not an e-mail server. [see [url]www.coolfusion.com[/url] and [url]www.lyris.com][/url] I guess the
blame for all this rests squarely on Micro$oft's shoulders. After trying to
destroy the browser market everyone seems to think every application should do
things it wasn't designed to do. (Dear Micro$oft: OS means Operating System
not Overbloated Software.) Anyone else miss the Apple II? C'est la vie Warm
Regards, Raven

[Spiralhead]

Wouldn't it have to download the emails off the pop server before it writes them to the DB?

[Tom Jordahl]

Try the (free) developers edition of CFMX 7 on your mailbox.

POP has been rewritten for CFMX 6, and further improved in 6.1 and 7. Still
has the character set problems described at the top of this thread however:
we can't read mail in charsets that Java doesn't support.

--
Tom Jordahl
Macromedia Server Development

[Wildcat_ut]

Welp I am just another of the growing population of cfpop problem having
persons. I have tried the cferror/cfcatch and its not catching a damn thing...
same error 'The cause of this exception was:
java.io.UnsupportedEncodingException: unicode-1-1-utf-7. ' and during the dump
I did a find for even UTF and its not finding it anywhere. Is there something
else in the header that would identify it to search for? Thanks.

[Paul WA]

If you are running on Windows, I have a couple of commercial CFX tags that
solve the encoding problems by using the native Windows encoding engine that is
built into the OS. It supports over 130 codepage conversions when your OS is
configured to do them....
[url]http://store.newmediadevelopment.net/cfdevelopment.cfm[/url]. The tags have been in
development for years and I have successfully run them on CF4.5, 5, MX, MX 6.1
and MX 7 on W2k and W2k3.

[Wildcat_ut]

What a freakin joke. I cannot believe that Macromedia has no work around for
this issue. I'm receiving 4-6 of these emails a month, which isnt alot out of
the thousands i'm receiving but its enough to reall piss ya off. If sun cant
pull their heads out of the dark spot, then use microsoft! I understand
strictly using the java engine helps with cross platform, but come on if it
dont work do something else at least for this part! KSmith, Nice idea one
major design flaw in it though. The message headers that return the error utf-7
error does not contain utf-7 anywhere in it. in fact it doesnt contain anything
close and I cant find any consistency between two. Here are two example
message headers that both caused this error: Message Header: Return-Path: <>
X-Real-To: Received: from [209.63.232.10] (HELO
spamkiller1.NetworldConnectionsInc.local) by networld.com (CommuniGate Pro SMTP
4.2.3) with ESMTP id 27276131 for [email]billing@networld.com[/email]; Sun, 03 Apr 2005
12:41:07 -0600 Received: from owen-cee.com (unverified [66.219.227.82]) by
spamkiller1.NetworldConnectionsInc.local(Vircom SMTPRS 4.0.340.0) with ESMTP id
for ;Sun, 3 Apr 2005 12:39:55 -0600 Message-ID: Date: Sun, 3 Apr 2005 12:40:56
-0600 From: Subject: Delivery Status Notification (Failure) To: X-DSNContext:
335a7efd - 4523 - 00000001 - 80040546 MIME-Version: 1.0 Content-Type:
multipart/report; boundary='9B095B5ADSN=_01C52051686DFF6A00002CBBowe n?cee.com'
Getting Message Number: 641
--------------------------------------------------------------------------------
Message Header: Return-Path: X-Real-To: Received: from [209.63.232.10] (HELO
spamkiller1.NetworldConnectionsInc.local) by networld.com (CommuniGate Pro SMTP
4.2.3) with ESMTP id 27393754; Tue, 05 Apr 2005 06:12:19 -0600 Received: from
c-67-174-245-122.hsd1.ca.comcast.net (unverified [67.174.245.122]) by
spamkiller1.NetworldConnectionsInc.local (Vircom SMTPRS 4.0.340.0) with SMTP id
; Tue, 5 Apr 2005 06:10:53 -0600 X-Message-Info:
A4FOAE8CTSneFCU63ioCFtfuH3C7ulUA6aHABtsJG19DTF57 Received: from mail pickup
service by 67.174.245.122 with Microsoft SMTPSVC; Tue, 05 Apr 2005 19:04:49
+0600 Content-Class: urn:content-classes:message Reply-To: 'Maurice Carrier'
From: 'Maurice Carrier' To: 'Ariccardi' Subject: Managed Care Director,over
440,000 listings of professional executives in the healthcare field. Date: Tue,
05 Apr 2005 14:02:49 +0100 MIME-Version: 1.0 Content-Type:
multipart/alternative; boundary='--212E4C653B6BCE91E50' Message-ID: Getting
Message Number: 733
--------------------------------------------------------------------------------
Now what I had to do to figure out which messages it was is do my initial
cfpop getheaders and loop through each one and instead of searching for 'utf-7'
I just did a cftry, cfpop getall, with the messagenumber. So it did a cfpop
try on every single message individually which was slow as crap but it was the
only way I could see to identify the messages and delete them. I cant put this
kinda crap on my production servers with as slow as it is. My biggest problem
is i'm using alot of 3rd party coldfusion aps that use the cfpop so now to fix
this stupid crap I have to decrypt every single page I get the error on change
and change someone elses code, which makes it a pain to just use a cfx tag.

[Spiralhead]

I hope someone at Macromedia is pressuring Sun to address this problem instead of just saying "Oh! it's not ColdFusion's fault--it's Java's fault!" Is anyone over at Sun even aware of the problem?

[Spiralhead]

Does anyone have experience using the CFX tags described above by Paul WA?
Specifically, these ones
here-->[url]http://store.newmediadevelopment.net/cfx_tag.cfm?ProductID=20[/url]

I mean, are they reliable for a heavilly trafficked email app?

[Neo Rye]

I just delete all problem emails. Not the best way, but then I don't have to
waist time trying to fix it. And I save the end user time because it's just one
less email to end up in their SPAM folder. <cftry> <!---start by getting the
mail in the account---> <cfpop action='getall' name='getEmailService'
attachmentpath='#GetDirectoryFromPath(GetCurrentTe mplatePath())#attachments\'
server='#Variables.MailServer#' timeout='300' username='#Variables.UserName#'
password='#Variables.Password#' generateuniquefilenames='yes'> <!---catch any
errors with cfpop---> <cfcatch> <!---error because of some issue with the
email---> <cfpop action='getheaderonly' name='runCheck'
server='#Variables.MailServer#' timeout='300' username='#Variables.UserName#'
password='#Variables.Password#' generateuniquefilenames='yes'> <!---loop
over our headers and try to find the one causing the problem---> <cfloop
query='runCheck'> <cftry> <!---get one message at a time and if it's
bad, delete it---> <cfpop action='getall' name='testEmail'
messagenumber='#runCheck.UID#'
attachmentpath='#GetDirectoryFromPath(GetCurrentTe mplatePath())#attachments\'
server='#Variables.MailServer#' timeout='300' username='#Variables.UserName#'
password='#Variables.Password#' generateuniquefilenames='yes'> <cfcatch>
<!---delete any bad emails---> <cfpop action='delete'
server='#Variables.MailServer#' username='#Variables.UserName#'
password='#Variables.Password#' uid='#runCheck.UID#'> </cfcatch>
</cftry> </cfloop> <!---now go back and get the cleaned emails--->
<cfpop action='getall' name='getEmailService'
attachmentpath='#GetDirectoryFromPath(GetCurrentTe mplatePath())#attachments\'
server='#Variables.MailServer#' timeout='300' username='#Variables.UserName#'
password='#Variables.Password#' generateuniquefilenames='yes'> </cfcatch>
</cftry>

[Tom Jordahl]

Actually, I lied - I *did* fix the charset issue in ColdFusion MX 7.

You will get the following in the TextBody column of the query for the
message with the unsupported encoding:
"Unable to retrieve message content: java.io.UnsupportedEncodingException:
utf-7 "

Where 'utf-7' is replaced by the encoding Java didn't support.

--
Tom Jordahl
Macromedia Server Development

[altimage]

I'm glad to see this issue coming up again. We have had this issue for years.
Like many people here, we're running into the issue trying to clean up bounced
emails. Our soultion was to write a bounce cleaner in php. It works very well.
The one big downside is that although I used to be very fluent in php, I feel a
bit rusty after having spend so many years with CF. For us, deleteing, skipping
or otherwise not handling the utf-7 messages is just not an option. Also since
we're on linux, the cfx tag wont work. I definitely agree with an earlier
poster's comment that this is a pretty unsatisfactory situation for a product
costing several thousand dollars. I would really like to see a hot fix on this
one. Email is too important not to get perfect.

AltImage