Issue
The problem
Suppose that I have written a lengthy script in some language "lang", and now want to convert this single-file script into a directory tree with a project consisting of many files. I want to insert some kind of separators and file-paths into this file, and process it in some way so that in the end I obtain:
- a proper project directory layout (sth. like this),
- build-definition file,
- readme's,
- separate subdirectories for
main/srcandtest/srcetc.
For example, given the following script (pseudocode):
// required dependencies, should be moved
// into the build definition build.foo
require "org.foo" % "foo-core" % "1.2.3"
require "org.bar" % "bar-gui" % "3.2.1"
// A longer comment that should be converted
// into a text file and moved into a 'notes'
// subdirectory
/*
#README
Another lengthy comment that should go into
a readme.md
*/
/** A class that should
* go to src/main/lang/proj/A.lang
*/
class A {
def a = "foo"
}
/** Another class
* that should go to src/main/lang/proj/B.lang
*/
class B {
def b = "bar"
}
/** Some tests,
* should end up in
* src/test/lang/proj/MyTest.lang
@Test def testFoo() {
assert(2 + 2 == 5)
}
and assuming that I can insert arbitrary separators, commands, escape-sequences and file paths into this file, I would like to obtain the following project:
project/
|-- build.txt
|-- notes
| `-- note_01.txt
|-- readme.md
`-- src
|-- main
| `-- lang
| `-- proj
| |-- A.lang
| `-- B.lang
`-- test
`-- lang
`-- proj
`-- MySpec.lang
Edit:
What follows is a less-sophisticated version of my own answer below
What I've tried
Here is one naive way to do it:
- Convert the original script into a bash script by prepending
#!/bin/bash - split the source code into HEREDOCS
- insert package declarations where necessary
- add bunch of
mkdir -pandcdbetween the HEREDOC-pieces catthe HEREDOC pieces into appropriately named files- test the script on empty directories until it works as expected
For the above script, it might look somehow like this:
#!/bin/bash
mkdir project
cd project
cat <<'EOF' > build.txt
// required dependencies, should be moved
// into the build definition build.foo
require "org.foo" % "foo-core" % "1.2.3"
require "org.bar" % "bar-gui" % "3.2.1"
EOF
mkdir notes
cd notes
cat <<'EOF' > note_01.txt
// A longer comment that should be converted
// into a text file and moved into a 'notes'
// subdirectory
EOF
cd ..
cat <<'EOF' > readme.md
/*
#README
Another lengthy comment that should go into
a readme.md
*/
EOF
mkdir -p src/main/lang/proj
cd src/main/lang/proj
cat <<'EOF' > A.lang
package proj
/** A class
* that should go to src/main/lang/proj/A.lang
*/
class A {
def a = "foo"
}
EOF
cat <<'EOF' > B.lang
package proj
/** Another class
* that should go to src/main/lang/proj/B.lang
*/
class B {
def b = "bar"
}
EOF
cd ../../..
mkdir -p test/lang/proj
cd test/lang/proj
cat <<'EOF' > MySpec.lang
package proj
/** Some tests,
* should end up in
* src/test/lang/proj/MyTest.lang
@Test def testFoo() {
// this should end up in test
assert(2 + 2 == 5)
}
EOF
cd ../../..
What's wrong with this approach
It does generate the correct tree, but this approach seems rather error-prone:
- it's too easy to
cd ../../..to the wrong nesting level - too easy to
mkdirwith a wrong name, and then fail tocdinto it. - There is no way to handle the entire tree construction as a single transaction, that is, if something fails later in the script, there is no simple way to clean up the mess generated before the error occurred.
I certainly could try to make it a bit less brittle by defining
special functions that mkdir and cd in one go, and
then wrap invocations of those functions together with cats into
(mkdirAndCd d ; cat) etc.
But it just doesn't feel quite right. Isn't there a much simpler
way to do it? Could one somehow combine the standard bash/linux utilities
into a tiny & very restricted domain specific language for
generating directory trees with text files? Maybe some newer version of split where one could specify where to split and where to put the pieces?
Related questions:
mkdirandtouchin single command- The reverse of
tree- reconstruct file and directory structure from text file contents?
Other interesting proposals that don't seem to work:
- Use tar. That would mean that one would have to convert the text file manually into a valid tar-archive. While a tar archive indeed is a single plain-text file, its internal format does not look like the most comfortable DSL for such a simple task. It was never intended to be used by humans directly in that way.
- Similar argument holds for
shar. Sincesharuses the bash itself to extract the archive, my above proposal is, in principle, a manually generated shar-archive in a very uncommon format, thereforesharseems to share all the drawbacks with the above proposal. I'd rather prefer something more restricted, that allows to do fewer things, but provides more guarantees about the quality of the outcome.
Maybe I should emphasize again that I don't have a tree to begin with, so there is nothing to compress. I have only the single script file and a rough idea of what the tree should look like in the end.
Solution
(my own answer)
Consider the following definition of a tiny embedded domain specific language for defining directory trees with text files:
#!/bin/bash
enter() {
local subdir="$1"
if [ ! -d "$subdir" ]
then
mkdir -p "$subdir"
fi
pushd "$subdir" > /dev/null
}
leave() {
popd > /dev/null
}
save() {
local fileName="$1"
cut -d'|' -f2- > "$fileName"
}
The enter command creates a directory if necessary, and cds into this directory, it works with arbitrary relative paths. The save command saves the text content of a here-document to file. The leave command changes to previous directory.
When a file is saved, the margin (empty space followed by '|') is stripped from each line. This is to ensure that the indentation of the script does not interfere with the indentation of the written files.
If these definitions are sourced, then the tree-generation script can be written as follows:
#!/bin/bash
source explode.sh
mkdir project
cd project
enter "src"
enter "main/lang/proj"
save "A.lang" <<'____EOF'
|package proj
|
|/** A totally useful class
| * that should go to src/main/lang/proj/A.lang
| */
|class A {
| def a = "foo"
|}
____EOF
save "B.lang" <<'____EOF'
|package proj
|/** Another very useful class
| * that should go to src/main/lang/proj/B.lang
| */
|class B {
| def b = "bar"
|}
____EOF
leave
enter "test/lang/proj"
save "MyTest.lang" <<'____EOF'
|package proj
|
|/** A test that should end up in
| * src/test/lang/proj/MyTest.lang
|@Test def testFoo() {
| assert(2 + 2 == 5)
|}
____EOF
leave
leave
save "build.txt" <<'EOF'
|require "org.foo" % "foo-core" % "1.2.3"
|require "org.bar" % "bar-gui" % "3.2.1"
EOF
enter "notes"
save "note_01.txt" <<'__EOF'
|A longer comment that should be converted
|into a text file and moved into a 'notes'
|subdirectory. This is a very long comment
|about the purpose of the project. Blah
|blah blah.
__EOF
leave
save "README.md" <<'EOF'
|#README
|
|This is a readme file for my awesome project.
|It ends with this line. Bye.
EOF
When executed, the script generates the following directory tree:
project/
├── build.txt
├── notes
│ └── note_01.txt
├── README.md
└── src
├── main
│ └── lang
│ └── proj
│ ├── A.lang
│ └── B.lang
└── test
└── lang
└── proj
└── MyTest.lang
The bash-script mirrors the tree structure very closely, and it's impossible to mess up the cd ../../../../../.. commands. It still lacks various desirable properties though (not transactional, no dry-run capability).
Answered By - Andrey Tyukin Answer Checked By - Pedro (WPSolving Volunteer)