Professional Web Applications Themes

Move a file - PERL Beginners

I am writing a script which will poll a directory and then move any files in the directory to a new directory dependant upon its name. The problem I think I might have is that the script may try to pick a file that is still being copied into the directory whilst the application is still trying to write the file. What is the best method in Perl to prevent errors. Cheers Neill ******************** IMPORTANT NOTICE This email (including any attachments) is meant only for the intended recipient. It may also contain confidential and privileged information. If you are not ...

  1. #1

    Default Move a file


    I am writing a script which will poll a directory and then move any files
    in the directory to a new directory dependant upon its name.

    The problem I think I might have is that the script may try to pick a file
    that is still being copied into the directory whilst the application is
    still trying to write the file.

    What is the best method in Perl to prevent errors.

    Cheers

    Neill










    ********************
    IMPORTANT NOTICE This email (including any attachments) is meant only for the intended recipient. It may also contain confidential and privileged information. If you are not the intended recipient, any reliance on, use, disclosure, distribution or copying of this email or attachments is strictly prohibited. Please notify the sender immediately by email if you have received this message by mistake and delete the email and all attachments.

    Any views or opinions in this email are solely those of the author and do not necessarily represent those of Trinity Mirror PLC or its associated group companies (hereinafter referred to as "TM Group"). TM Group accept no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Although every reasonable effort is made to keep its network free from viruses, TM Group accept no liability for any virus transmitted by this email or any attachments and the recipient should use up-to-date virus checking software. Email to or from this address may be subject to interception or monitoring for operational reasons or for lawful business practices.

    Trinity Mirror PLC is the parent company of the Trinity Mirror group of companies and is registered in England No 82548, with its address at One Canada Square, Canary Wharf, London E14 5AP.
    ********************
    Neill Taylor Guest

  2. #2

    Default Re: Move a file

    Neill Taylor wrote:
    >
    > I am writing a script which will poll a directory and then move any files
    > in the directory to a new directory dependant upon its name.
    >
    > The problem I think I might have is that the script may try to pick a file
    > that is still being copied into the directory whilst the application is
    > still trying to write the file.
    >
    > What is the best method in Perl to prevent errors.
    Hi Neill.

    If your application is well-behaved then it may lock the file
    for writing. If your Perl program asks for an exclusive lock
    then it will be granted if it no longer being accessed.
    Take a look at

    perldoc -f flock

    Another way is to check the size of file. A file will often
    show a size of zero until it is closed by the writing
    application.

    Finally, if only one file at a time is being written then
    just sort the them in order of creation date and move all
    but the latest one.

    What works you will find only by experimentation.

    HTH,

    Rob


    Rob Dixon Guest

  3. #3

    Default Re: Move a file


    > Neill Taylor wrote:
    > >
    > > I am writing a script which will poll a directory and then move any
    files
    > > in the directory to a new directory dependant upon its name.
    > >
    > > The problem I think I might have is that the script may try to pick
    a file
    > > that is still being copied into the directory whilst the application is
    > > still trying to write the file.
    > >
    > > What is the best method in Perl to prevent errors.
    >
    > Hi Neill.
    >
    > If your application is well-behaved then it may lock the file
    > for writing. If your Perl program asks for an exclusive lock
    > then it will be granted if it no longer being accessed.
    > Take a look at
    >
    > perldoc -f flock
    >
    > Another way is to check the size of file. A file will often
    > show a size of zero until it is closed by the writing
    > application.
    >
    > Finally, if only one file at a time is being written then
    > just sort the them in order of creation date and move all
    > but the latest one.
    >
    > What works you will find only by experimentation.
    >
    Depending on what is writing the files, aka another controllable
    program? I have had good luck with writing the file to a temporary
    location, usually with a dot on the front, then executing a move from
    that temp location to the real name. A move is usually a single action
    (at least that's my understanding) and for most filesystems is merely a
    single inode update. Then just have the directory watcher skip dot
    files... Though the lock would probably be better...

    [url]http://danconia.org[/url]

    --
    Boycott the Sugar Bowl! You couldn't pay me to watch that game.
    Wiggins D Anconia Guest

  4. #4

    Default Re: Move a file

    Wiggins D Anconia wrote:
    >
    > > Neill Taylor wrote:
    > > >
    > > > I am writing a script which will poll a directory and then move any
    > files
    > > > in the directory to a new directory dependant upon its name.
    > > >
    > > > The problem I think I might have is that the script may try to pick
    > a file
    > > > that is still being copied into the directory whilst the application is
    > > > still trying to write the file.
    > > >
    > > > What is the best method in Perl to prevent errors.
    > >
    > > Hi Neill.
    > >
    > > If your application is well-behaved then it may lock the file
    > > for writing. If your Perl program asks for an exclusive lock
    > > then it will be granted if it no longer being accessed.
    > > Take a look at
    > >
    > > perldoc -f flock
    > >
    > > Another way is to check the size of file. A file will often
    > > show a size of zero until it is closed by the writing
    > > application.
    > >
    > > Finally, if only one file at a time is being written then
    > > just sort the them in order of creation date and move all
    > > but the latest one.
    > >
    > > What works you will find only by experimentation.
    > >
    >
    > Depending on what is writing the files, aka another controllable
    > program? I have had good luck with writing the file to a temporary
    > location, usually with a dot on the front, then executing a move from
    > that temp location to the real name. A move is usually a single action
    > (at least that's my understanding) and for most filesystems is merely a
    > single inode update. Then just have the directory watcher skip dot
    > files... Though the lock would probably be better...
    Hi Wiggins.

    Yes, that will often work, depending on the filing system. But a 'move'
    will usually do 'copy' followed by 'delete'. Occasionally you can move
    a file while it is being written to, as the application doing the writing
    doesn't care where it appears in the directory structure. But that's
    unusual. It's more common that the 'copy' will be allowed to grab an
    image of the data flushed so far and the subsequent 'delete' will
    be executed once the writing app has closed the original file.

    Without extensive knowledge of the platform and filing system I don't
    think it's safe to assume anything!

    Cheers,

    Rob


    Rob Dixon Guest

  5. #5

    Default RE: Move a file

    A variation of Wiggins' approach is to pre or append the PID to the file
    name. This can potentially allow multiple processes of the same type to run
    simultaneously without stomping on each other while they are running and can
    aid in tracking down those production problems - assuming you are logging
    your PID, or just figuring out which zombie process owns which file...

    it's a little more work to ignore, or you can use both ideas at the same
    time and ignore files starting with dot.

    -Tom Kinzer

    -----Original Message-----
    From: Wiggins d Anconia [mailto:wigginsdanconia.org]
    Sent: Friday, December 12, 2003 6:37 AM
    To: [email]beginnersperl.org[/email]
    Subject: Re: Move a file



    > Neill Taylor wrote:
    > >
    > > I am writing a script which will poll a directory and then move any
    files
    > > in the directory to a new directory dependant upon its name.
    > >
    > > The problem I think I might have is that the script may try to pick
    a file
    > > that is still being copied into the directory whilst the application is
    > > still trying to write the file.
    > >
    > > What is the best method in Perl to prevent errors.
    >
    > Hi Neill.
    >
    > If your application is well-behaved then it may lock the file
    > for writing. If your Perl program asks for an exclusive lock
    > then it will be granted if it no longer being accessed.
    > Take a look at
    >
    > perldoc -f flock
    >
    > Another way is to check the size of file. A file will often
    > show a size of zero until it is closed by the writing
    > application.
    >
    > Finally, if only one file at a time is being written then
    > just sort the them in order of creation date and move all
    > but the latest one.
    >
    > What works you will find only by experimentation.
    >
    Depending on what is writing the files, aka another controllable
    program? I have had good luck with writing the file to a temporary
    location, usually with a dot on the front, then executing a move from
    that temp location to the real name. A move is usually a single action
    (at least that's my understanding) and for most filesystems is merely a
    single inode update. Then just have the directory watcher skip dot
    files... Though the lock would probably be better...

    [url]http://danconia.org[/url]

    --
    Boycott the Sugar Bowl! You couldn't pay me to watch that game.

    --
    To unsubscribe, e-mail: [email]beginners-unsubscribeperl.org[/email]
    For additional commands, e-mail: [email]beginners-helpperl.org[/email]
    <http://learn.perl.org/> <http://learn.perl.org/first-response>


    Tom Kinzer Guest

  6. #6

    Default Re: Move a file

    Rob Dixon wrote:
    > Wiggins D Anconia wrote:
    >
    <snip>
    >>
    >>Depending on what is writing the files, aka another controllable
    >>program? I have had good luck with writing the file to a temporary
    >>location, usually with a dot on the front, then executing a move from
    >>that temp location to the real name. A move is usually a single action
    >>(at least that's my understanding) and for most filesystems is merely a
    >>single inode update. Then just have the directory watcher skip dot
    >>files... Though the lock would probably be better...
    >
    >
    > Hi Wiggins.
    >
    > Yes, that will often work, depending on the filing system. But a 'move'
    > will usually do 'copy' followed by 'delete'.
    What filesystems act in that matter? I believe you that there are ones
    that do it that way, none of the Unix variants that I have experienced
    do (or seem to). Unless we are talking about 'mv' as opposed to
    'rename' which I should have been more explicit about. A rename merely
    moves the starting inode, assuming the file is staying on the same
    partition, which in this case if the temporary file is written into the
    same directory then it is...

    Occasionally you can move
    > a file while it is being written to, as the application doing the writing
    > doesn't care where it appears in the directory structure. But that's
    > unusual. It's more common that the 'copy' will be allowed to grab an
    > image of the data flushed so far and the subsequent 'delete' will
    > be executed once the writing app has closed the original file.
    >
    > Without extensive knowledge of the platform and filing system I don't
    > think it's safe to assume anything!
    >
    Right which is why the locking is better. However, in many cases one
    will be familar enough with the system where the code is deployed, so
    this method is just shorter (to code).

    [url]http://danconia.org[/url]

    Wiggins D'Anconia Guest

  7. #7

    Default Re: Move a file

    Hi guys,
    I am not a programmer but I would like to get a script that can sort files depending of the fact that their names are listed in a txt file or not.
    So I would have a folder with all files, a new folder for some of the files, and a txt file with a list of the files I want in the new folder.
    I can launch Perl on my computer (Windows).

    I tried writing something like:

    use File::Copy;
    if (ARGV != 3) {
    print STDOUT "Usage: script [FileNAme] [SourcePath] [MovePath]\n";
    exit;
    }
    open($FileName,"<ARGV[0]"); #File with list of file names
    open(SourcePath,"<ARGV[1]"); #Current path that we copy from
    open(MovePath,"<ARGV[2]"); #New path that we copy to
    my Filename;
    while ($line = <FileName> ) {
    chomp $line;
    push (FileName, split(/\t/,$line));
    }
    foreach $FileName(FileName){
    system(copy SourcePath /"FileName", MovePath/"FileName");
    }
    close($FileName);

    But I don't really know what I am doing…
    Does anyone with real skills could help me to write such script?
    Thanks a lot!
    Unregistered Guest

Similar Threads

  1. Move pages inside the PDF file
    By isup@adobeforums.com in forum Adobe Acrobat Macintosh
    Replies: 2
    Last Post: August 3rd, 01:49 AM
  2. Move on open file - not updated
    By Murray *TMM* in forum Dreamweaver AppDev
    Replies: 1
    Last Post: March 17th, 04:34 PM
  3. file.move to unc causes error
    By Michael McLaughlin in forum ASP.NET Security
    Replies: 5
    Last Post: December 29th, 03:30 AM
  4. Replies: 0
    Last Post: October 31st, 02:02 AM
  5. #25977 [NEW]: fopen('file', 'r'); doesn't move to the end of the file
    By nightstorm at tlen dot pl in forum PHP Development
    Replies: 0
    Last Post: October 24th, 04:53 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139