ActiveState XML-DOM 1.43

[Markus Mohr]

Hi, all,

when it comes to parsing large files into something XML readable, a
high amount of memory is required, and the CPU rate of the computer
(even with 1 GB RAM) goes up to almost 100 %.
Is there something like a switch or any other means to steer the
memory behaviour of this module?

Sincerely


Markus Mohr

P. S.:

Here is some sample code of how I do parse my external file (one at a
time). By this code, I parse an external file which is built up
according to the CDA standard (Clinical Document Architecture). This
file contains an uuencoded / uudecoded image object:

#------------------------------------------------------------------------------#
# Subroutine, um eine Anfrage in eine Patientenakte umzuwandeln
#
#------------------------------------------------------------------------------#
sub import_anfrage ($$) {
my ( $self, $anfrage, $konfiguration ) = @_;
print "Die ANFRAGE wird imporiert:\n";
print "---------------------------\n";

open( TEMP, ">./anf_temp.anf" );
print TEMP $anfrage;
close TEMP;

# Wir legen ein neues XML-Objekt an, das alte wird verworfen
my $xml = CXML->new();
$xml->construct_xml($konfiguration);
$xml = $konfiguration->get_value('xml');
my $xml_root = $xml->{'root'};

# Die Anfrage wird in ein XML-Dokument geparst
my $anfrage_parser = new XML::DOM::Parser( KeepCDATA => 1 );
my $anfrage_doc = $anfrage_parser->parsefile("./anf_temp.anf");
unlink("./anf_temp.anf");

# Die Anfrage ist Teil der neuen EPA
my $anfrage_root = $anfrage_doc->getElementsByTagName('ANFRAGE');
$anfrage_root = $anfrage_root->item(0);
$anfrage_root->setOwnerDocument( $xml->{'doc'} );
my $nodes = $xml_root->getElementsByTagName('anfragen');
my $node = $nodes->item(0);
$node->appendChild($anfrage_root);

# Anschliessend die Daten der Anfrage in die EPA übertragen
foreach my $type (qw( PATIENT ARZT INSTITUTION UNTERSUCHUNG
DIAGNOSE ANAMNESE MASSNAHME soziomedizinischedaten )) {
my $anfrage = $xml_root->getElementsByTagName('ANFRAGE');
$anfrage = $anfrage->item(0);
for my $element ( $anfrage->getElementsByTagName($type) ) {
my $destination_parent;
SWITCH: for ($type) {
/PATIENT/ && do { $destination_parent =
'patient'; last; };
/ARZT/ && do { $destination_parent =
'arztliste'; last; };
/INSTITUTION/ && do { $destination_parent =
'paramedizinischeliste'; last; };
/UNTERSUCHUNG/ && do { $destination_parent =
'untersuchungen'; last; };
/DIAGNOSE/ && do { $destination_parent =
'diagnosen'; last; };
/ANAMNESE/ && do { $destination_parent =
'anamnesen'; last; };
/MASSNAHME/ && do { $destination_parent =
'massnahmen'; last; };

#/soziomedi/ && do { $destination_parent =
$xml_root; last; };
}

$destination_parent =
$xml_root->getElementsByTagName($destination_parent);
$destination_parent = $destination_parent->item(0);
my $source = $element->cloneNode(1);

$destination_parent = $xml_root if ( $type =~ /soziomed/
);
# print "Vorher:\n",
$destination_parent->toString, "\n";
print "TYPE: $type DP $destination_parent CT ",
$source->toString, "\n";

$destination_parent->appendChild($source);
$destination_parent->removeChild(
$destination_parent->getElementsByTagName('soziomedizinischedaten')->item(0)
) if ( $type =~ /soziomed/ );
# print "\nJetzt:\n", $destination_parent->toString, "\n";

# Bei den Daten des Patienten die alten (=leeren) Daten
entfernen
if ( $type eq 'PATIENT' ) {
my $old_data = $destination_parent->getFirstChild;
$destination_parent->removeChild($old_data);
}

# Altes Element aus der Anfrage entfernen
my $source_parent = $element->getParentNode();
$source_parent->removeChild($element);
}
}

return;
}

[Markus Mohr]

On Fri, 04 Jun 2004 16:05:45 +0200, Markus Mohr
<markus.mohr@mazimoi.de> wrote:

>Hi, all,
>
>when it comes to parsing large files into something XML readable, a
>high amount of memory is required, and the CPU rate of the computer
>(even with 1 GB RAM) goes up to almost 100 %.
>Is there something like a switch or any other means to steer the
>memory behaviour of this module?
>
>Sincerely
>
>
>Markus Mohr
>
>P. S.:
>
>Here is some sample code of how I do parse my external file (one at a
>time). By this code, I parse an external file which is built up
>according to the CDA standard (Clinical Document Architecture). This
>file contains an uuencoded / uudecoded image object:
>
>#------------------------------------------------------------------------------#
># Subroutine, um eine Anfrage in eine Patientenakte umzuwandeln
>#
>#------------------------------------------------------------------------------#
>sub import_anfrage ($$) {
> my ( $self, $anfrage, $konfiguration ) = @_;
> print "Die ANFRAGE wird imporiert:\n";
> print "---------------------------\n";
>
> open( TEMP, ">./anf_temp.anf" );
> print TEMP $anfrage;
> close TEMP;
>
> # Wir legen ein neues XML-Objekt an, das alte wird verworfen
> my $xml = CXML->new();
> $xml->construct_xml($konfiguration);
> $xml = $konfiguration->get_value('xml');
> my $xml_root = $xml->{'root'};
>
> # Die Anfrage wird in ein XML-Dokument geparst
> my $anfrage_parser = new XML::DOM::Parser( KeepCDATA => 1 );
> my $anfrage_doc = $anfrage_parser->parsefile("./anf_temp.anf");
> unlink("./anf_temp.anf");
>
> # Die Anfrage ist Teil der neuen EPA
> my $anfrage_root = $anfrage_doc->getElementsByTagName('ANFRAGE');
> $anfrage_root = $anfrage_root->item(0);
> $anfrage_root->setOwnerDocument( $xml->{'doc'} );
> my $nodes = $xml_root->getElementsByTagName('anfragen');
> my $node = $nodes->item(0);
> $node->appendChild($anfrage_root);
>
> # Anschliessend die Daten der Anfrage in die EPA übertragen
> foreach my $type (qw( PATIENT ARZT INSTITUTION UNTERSUCHUNG
>DIAGNOSE ANAMNESE MASSNAHME soziomedizinischedaten )) {
> my $anfrage = $xml_root->getElementsByTagName('ANFRAGE');
> $anfrage = $anfrage->item(0);
> for my $element ( $anfrage->getElementsByTagName($type) ) {
> my $destination_parent;
> SWITCH: for ($type) {
> /PATIENT/ && do { $destination_parent =
>'patient'; last; };
> /ARZT/ && do { $destination_parent =
>'arztliste'; last; };
> /INSTITUTION/ && do { $destination_parent =
>'paramedizinischeliste'; last; };
> /UNTERSUCHUNG/ && do { $destination_parent =
>'untersuchungen'; last; };
> /DIAGNOSE/ && do { $destination_parent =
>'diagnosen'; last; };
> /ANAMNESE/ && do { $destination_parent =
>'anamnesen'; last; };
> /MASSNAHME/ && do { $destination_parent =
>'massnahmen'; last; };
>
> #/soziomedi/ && do { $destination_parent =
>$xml_root; last; };
> }
>
> $destination_parent =
>$xml_root->getElementsByTagName($destination_parent);
> $destination_parent = $destination_parent->item(0);
> my $source = $element->cloneNode(1);
>
> $destination_parent = $xml_root if ( $type =~ /soziomed/
>);
> # print "Vorher:\n",
>$destination_parent->toString, "\n";
> print "TYPE: $type DP $destination_parent CT ",
>$source->toString, "\n";
>
> $destination_parent->appendChild($source);
> $destination_parent->removeChild(
>$destination_parent->getElementsByTagName('soziomedizinischedaten')->item(0)
>) if ( $type =~ /soziomed/ );
> # print "\nJetzt:\n", $destination_parent->toString, "\n";
>
> # Bei den Daten des Patienten die alten (=leeren) Daten
>entfernen
> if ( $type eq 'PATIENT' ) {
> my $old_data = $destination_parent->getFirstChild;
> $destination_parent->removeChild($old_data);
> }
>
> # Altes Element aus der Anfrage entfernen
> my $source_parent = $element->getParentNode();
> $source_parent->removeChild($element);
> }
> }
>
> return;
>}

Most probably there is also a problem with the concepts of puudecode
and puuencode I use. They stem from the work of Andreas J. König in
the underlying Convert::UU module, but seem to be very slow.

Any idea how to prevent this slowness?

Sincerely


Markus Mohr