Thursday, October 27, 2022

[SOLVED] shell script to create multiple files, incrementing from last file upon next execution

Issue

I'm trying to create a shell script that will create multiple files (or a batch of files) of a specified amount. When the amount is reached, script stops. When the script is re-executed, the files pick up from the last file created. So if the script creates files 1-10 on first run, then on the next script execution should create 11-20, and so on.

enter code here
#!/bin/bash
NAME=XXXX
valid=true
NUMBER=1
while [ $NUMBER -le 5 ];
do
   touch $NAME$NUMBER
   ((NUMBER++))
   echo $NUMBER + "batch created"
   if [ $NUMBER == 5 ];
   then
     break
   fi
   touch $NAME$NUMBER
   ((NUMBER+5))
   echo "batch complete"
done

Solution

Based on my comment above and your description, you can write a script that will create 10 numbered files (by default) each time it is run, starting with the next available number. As mentioned, rather than just use a raw-unpadded number, it's better for general sorting and listing to use zero-padded numbers, e.g. 001, 002, ...

If you just use 1, 2, ... then you end up with odd sorting when you reach each power of 10. Consider the first 12 files numbered 1...12 without padding. a general listing sort would produce:

file1
file11
file12
file2
file3
file4
...

Where 11 and 12 are sorted before 2. Adding leading zeros with printf -v avoids the problem.

Taking that into account, and allowing the user to change the prefix (first part of the file name) by giving it as an argument, and also change the number of new files to create by passing the count as the 2nd argument, you could do something like:

 #!/bin/bash

prefix="${1:-file_}"                ## beginning of filename
number=1                            ## start number to look for
ext="txt"                           ## file extension to add

newcount="${2:-10}"                 ## count of new files to create

printf -v num "%03d" "$number"      ## create 3-digit start number
fname="$prefix$num.$ext"            ## form first filename

while [ -e "$fname" ]; do           ## while filename exists
  number=$((number + 1))            ## increment number
  printf -v num "%03d" "$number"    ## form 3-digit number
  fname="$prefix$num.$ext"          ## form filename
done

while ((newcount--)); do            ## loop newcount times
  touch "$fname"                    ## create filename
  ((! newcount)) && break;          ## newcount 0, break (optional)
  number=$((number + 1))            ## increment number
  printf -v num "%03d" "$number"    ## form 3-digit number
  fname="$prefix$num.$ext"          ## form filename
done

Running the script without arguments will create the first 10 files, file_001.txt - file_010.txt. Run a second time, it would create 10 more files file_011.txt to file_020.txt.

To create a new group of 5 files with the prefix of list_, you would do:

bash scriptname list_ 5

Which would result in the 5 files list_001.txt to list_005.txt. Running again with the same options would create list_006.txt to list_010.txt.

Since the scheme above with 3 digits is limited to 1000 files max (if you include 000), there isn't a big need to get the number from the last file written (bash can count to 1000 quite fast). However, if you used 7-digits, for 10 million files, then you would want to parse the last number with ls -1 | tail -n 1 (or version sort and choose the last file). Something like the following would do:

  number=$(ls -1 "$prefix"* | tail -n 1 | grep -o '[1-9][0-9]*')

(note: that is ls -(one) not ls -(ell))

Let me know if that is what you are looking for.



Answered By - David C. Rankin
Answer Checked By - Dawn Plyler (WPSolving Volunteer)
at