Heavy number of collection to create

[francois-yanick]

I try to create several search collections from 3 indented loops that gives me
an approximative amount of 234 search collections. I am doing this because I do
not know how many search collection I will have to create an advance so I have
to do it dynamically.

But, it seems that the number of search collection to create is too much. I am
able to create 47 search collections on 234. This is not doing the job. Also, I
am a newbie in Using search collection with ColdFusion but I wonder if there is
a possibility to check before to create if a Search Collection already exist.
With that, I might be able to bypass the one (search collection) which are
already created and jumb to the next search collection to create. Maybe I am
not thinking in the right direction and I will appreciate any comment about my
idea!

I am using ColdFusion MX 7 Standard Edition on a Windows 2000 Server with 1GB
of RAM. I checked the capacity of the hard disk and it should be okay for the
234 search collections because I have plenty of space (more than 7 GB free for
that partition).

So this is the message I got:

An error occurred while registering the collection with the K2 Search server:
com.verity.api.administration.ConfigurationExcepti on: Out of memory. (-22)

I also attached the code in case somebody want to know from the code point of
view!

Thank's for any help!



<!--- /corpo2/search/scripts/CreateSearchCollectionsPublications.cfm --->

<!--- Collections pour les publications --->

<cfoutput><font size="+2"><b>ALPHABETICAL ORDER</b></font><hr></cfoutput>

<!--- ALPHABETICAL ORDER --->

<cfquery name="qTopics" datasource="#Application.DSN#">
SELECT *
FROM corpo2_Topic
</cfquery>
<cfloop index="cptTopic" from="1" to="#qTopics.RecordCount#">
<cfquery name="qDocumenttype" datasource="#Application.DSN#">
SELECT *
FROM corpo2_DocumentType
</cfquery>

<cfloop index="cptDocType" from="1" to="#qDocumenttype.RecordCount#">
<cfloop index="cptLang" from="1" to="3">
<cfcollection action="create"
collection="scPublicationsAlphaT#cptTopic#DT#cptDo cType#L#cptLang#"
path="g:\corpo2\search\" categories="yes">
<cfoutput>La collection de recherche
scPublicationsAlphaT#cptTopic#DT#cptDocType#L#cptL ang# a ?t? cr??<hr></cfoutput>
</cfloop>
</cfloop>
</cfloop>

<cfoutput><font size="+2"><b>CHRONOLOGICAL ORDER</b></font><hr></cfoutput>

<!--- CHRONOLOGICAL ORDER --->

<cfquery name="qTopics" datasource="#Application.DSN#">
SELECT *
FROM corpo2_Topic
</cfquery>
<cfloop index="cptTopic" from="1" to="#qTopics.RecordCount#">
<cfquery name="qDocumenttype" datasource="#Application.DSN#">
SELECT *
FROM corpo2_DocumentType
</cfquery>

<cfloop index="cptDocType" from="1" to="#qDocumenttype.RecordCount#">
<cfloop index="cptLang" from="1" to="3">
<cfcollection action="create"
collection="scPublicationsChronoT#cptTopic#DT#cptD ocType#L#cptLang#"
path="g:\corpo2\search\" categories="yes">
<cfoutput>La collection de recherche
scPublicationsChronoT#cptTopic#DT#cptDocType#L#cpt Lang# a ?t?
cr??<hr></cfoutput>
</cfloop>
</cfloop>
</cfloop>

[francois-yanick]

I try to understand the example that the official ColdFusion MX 7 gives about
how to verify if a collection already exist:

REF :
[url]http://livedocs.macromedia.com/coldfusion/7/htmldocs/wwhelp/wwhimpl/common/html/[/url]
wwhelp.htm?context=ColdFusion_Documentation&file=0 0000230.htm

I tried this code (see attached code) go throught the code for not having
found the name either if the collection already exist. Of course, this gives an
error to mention that the collection already exist and cannot be created again.

There is also another explication that says that if we try to recreate a
collection already exist, the system will map it instead of trying to recreate
it. I am confuse by this one.

[from the page I gave as reference...]
create: registers the collection with ColdFusion.
If the collection is present: creates a map to it.
If the collection is not present: creates it.



<!--- /corpo2/search/scripts/CreateSearchCollectionsPublications.cfm --->

<!--- Collections pour les publications --->

<cfoutput><font size="+2"><b>ALPHABETICAL ORDER</b></font><hr></cfoutput>

<!--- ALPHABETICAL ORDER --->

<cfquery name="qTopics" datasource="#Application.DSN#">
SELECT *
FROM corpo2_Topic
</cfquery>
<cfloop index="cptTopic" from="1" to="#qTopics.RecordCount#">
<cfquery name="qDocumenttype" datasource="#Application.DSN#">
SELECT *
FROM corpo2_DocumentType
</cfquery>

<cfloop index="cptDocType" from="1" to="#qDocumenttype.RecordCount#">
<cfloop index="cptLang" from="1" to="3">
<cfcollection action="list" name="myCollections" >
<cfquery name="qoq" dbtype="query">select * from myCollections where
myCollections.name =
'scPublicationsAlphaT#cptTopic#DT#cptDocType#L#cpt Lang#'</cfquery>
<cfif qoq.recordcount EQ 0>
<!--- Collection not exists --->
<cfcollection action="create"
collection="scPublicationsAlphaT#cptTopic#DT#cptDo cType#L#cptLang#"
path="g:\corpo2\search\" categories="yes">
<cfoutput>La collection de recherche
scPublicationsAlphaT#cptTopic#DT#cptDocType#L#cptL ang# a ?t? cr??<hr></cfoutput>
</cfif>
</cfloop>
</cfloop>
</cfloop>

<cfoutput><font size="+2"><b>CHRONOLOGICAL ORDER</b></font><hr></cfoutput>

<!--- CHRONOLOGICAL ORDER --->

<cfquery name="qTopics" datasource="#Application.DSN#">
SELECT *
FROM corpo2_Topic
</cfquery>
<cfloop index="cptTopic" from="1" to="#qTopics.RecordCount#">
<cfquery name="qDocumenttype" datasource="#Application.DSN#">
SELECT *
FROM corpo2_DocumentType
</cfquery>

<cfloop index="cptDocType" from="1" to="#qDocumenttype.RecordCount#">
<cfloop index="cptLang" from="1" to="3">
<cfcollection action="list" name="myCollections" >
<cfquery name="qoq" dbtype="query">select * from myCollections where
myCollections.name =
'scPublicationsChronoT#cptTopic#DT#cptDocType#L#cp tLang#'</cfquery>
<cfif qoq.recordcount EQ 0>
<!--- Collection not exists --->
<cfcollection action="create"
collection="scPublicationsChronoT#cptTopic#DT#cptD ocType#L#cptLang#"
path="g:\corpo2\search\" categories="yes">
<cfoutput>La collection de recherche
scPublicationsChronoT#cptTopic#DT#cptDocType#L#cpt Lang# a ?t?
cr??<hr></cfoutput>
</cfif>
</cfloop>
</cfloop>
</cfloop>

[francois-yanick]

I made a mistake when I mentioned 234 collections. Actually, just for this
particular need, I need to create 468 collections. I put this information in
case there is some limitation issues I should know! But I personnally don't
think it should affect the possibility to do it!! ;-))

[JimGroves]

Unfortunately they snuck in a wildy more memory intensive version of verity in
MX 7. I am sure they will extole the virtues of having a separate instance of
verity, thereby caching results, etc. but the reality is it takes ridiculous
amounts of memory compared to 6.1

[francois-yanick]

Ohhh! This is not good news!!! Well, I will have to find a way to built the
search collection without to being in a loop in a loop in a loop to safe to
safe some memory!!! As far as I understand this issue (memory), the system will
keep the process in memory until it terminate the loop and in my case, I have
three generations of loop.

I was able to use the categories to skip one loop so I have now two
generations of loops but it still asking too much RAM. I hate to have this kind
of problem!!! but nothing is impossible!!

[robstacey]

Just something that I leanred....k2 (under MX7) will only allow 128 collections. So, that might be part of your problem.