WOODY'S
FINDINGS

What's new in Scout 3.0.0?

Checkout the project page

The version 3.0.0 can be downloaded on the releases page

Summary
  1. Array subscripting with negative indexes
  2. Data formats conversion
  3. List paths
  4. Scripting recipes

The examples will be given for the following JSON, available in the Playground folder (you might want to download it to keep it in sight). The same commands are available for Plist, YAML and XML data.

{
  "Tom" : {
    "age" : 68,
    "hobbies" : [
      "cooking",
      "guitar"
    ],
    "height" : 175
  },
  "Robert" : {
    "age" : 23,
    "hobbies" : [
      "video games",
      "party",
      "tennis"
    ],
    "running_records" : [
      [
        10,
        12,
        9,
        10
      ],
      [
        9,
        12,
        11
      ]
    ],
    "height" : 181
  },
  "Suzanne" : {
    "job" : "actress",
    "movies" : [
      {
        "title" : "Tomorrow is so far",
        "awards" : "Best speech for a silent movie"
      },
      {
        "title" : "Yesterday will never go",
        "awards" : "Best title"
      },
      {
        "title" : "What about today?"
      }
    ]
  }
}
 

Array subscripting with negative indexes

The library now offers to subscript an array with a negative index. This figure gives an example with the 'ducks' array.

["Riri", "Fifi", "Loulou", "Donald", "Daisy"]
[  0   ,   1   ,    2    ,    3    ,    4   ] (Positive)
[ -5   ,  -4   ,   -3    ,   -2    ,   -1   ] (Negative)

Set the title of the second movie starting from the end in Suzanne's movies array. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout set -i People.plist "Suzanne.movies[-2].title=Beautiful night to die"
<p class="form-language-alternative-label">Swift</p>
try plist.set("Suzanne", "movies", -2, to: "Beautiful night to die")


Adding a value with set and -1 Breaking change

Appending a new value at the end of an array with the set command and the index -1 is no longer possible because of the new negative index subscripting feature. Thus, the following command does not add a new value at the end of the Tom's hobbies array but set the last value to the new one. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout set -i People.yml "Tom.hobbies[-1]=drawing"
<p class="form-language-alternative-label">Swift</p>
try yaml.set("Tom", "hobbies", -1, to: "Drawing")
To add a value at the end of an array, the command add is now the one to use, with the count (count|[#]) element. So, to add the new hobby to Tom's hobbies array, the following command can be used. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>
scout add -i People.yml "Tom.hobbies[#]=drawing"
<p class="form-language-alternative-label">Swift</p>
try yaml.add("drawing", at: "Tom", "hobbies", .count)

Array slicing Breaking change

The array slicing feature has also been updated to reflect the new subscripting with negative indexes. With the 'ducks' array exposed above, here are some examples.

Read Suzanne's last two movies titles. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>
scout read -i People.xml "Suzanne.movies[-2:].title"
<p class="form-language-alternative-label">Swift</p>
try xml.get("Suzanne","movies", .slice(-2, .last))


Data formats conversion

It's possible to convert the data to another format with the commands read, set, delete and add. The option -e|--export will output the data as the specified format.

About the conversion from XML

The conversion from XML can change the data structure when a tag has one ore more attributes. In such a case, the key will be transformed to a dictionary with two keys: "attributes" and "value". The "attribute" key will be a dictionary holding the attributes and the "value" key will hold the value of the key.

Output the file People.json as YAML. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout read -i People.json -e yaml
<p class="form-language-alternative-label">Swift</p>
try json.exportDataTo(.yaml)

Set Tom age to 40 and export the modified data to XML

<form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>
scout set "Tom.age=40" /
-i People.plist -e xml
<p class="form-language-alternative-label">Swift</p>
try plist.set("Tom", "age", to: 40)
try plist.exportData(to: .xml)


List paths

This features allows to list all the paths in the data. It can be useful to iterate over the values in a shell script and then play with the read, set, delete and add commands.

List all the paths in the file People.plist. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.plist
<p class="form-language-alternative-label">Swift</p>
try plist.listPaths()

Target single or group values

When listing paths, it's possible to target only single values (e.g. string, number...), group values (e.g. array, dictionary) or both. The default target is both single and group.

List all the paths leading to single values in the file People.xml. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.xml --single
<p class="form-language-alternative-label">Swift</p>
try xml.listPaths(filter: .targetOnly(.single))

List all the paths leading to group values in the file People.xml. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.xml --group
<p class="form-language-alternative-label">Swift</p>
try xml.listPaths(filter: .targetOnly(.group))

Initial path

Optionally provide a path from which the paths should be listed. The special elements like array slices or dictionary keys filters are supported

List all the paths in the file People.yml in Robert dictionary. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths "Robert" -i People.yml
<p class="form-language-alternative-label">Swift</p>
try yaml.listPaths(startingAt: "Robert")

List all the paths in the file People.yml in Robert and Tom dictionary. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.yml "#Robert|Tom#"
<p class="form-language-alternative-label">Swift</p>
try yaml.listPaths(startingAt: .filter("Robert|Tom"))

List all the paths leading to Suzanne's movies titles in the file People.yml. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.yml "Suzanne.movies[:].title"
<p class="form-language-alternative-label">Swift</p>
try yaml.listPaths(startingAt: "Suzanne", "movies", .slice(.first, .last), "title")

Filter the keys

It's possible to provide a regular expression to filter the paths final key. Only the paths whose final value is validated by the regular expression will be retrieved.
(The Swift examples use convenience initialisers).

List all the paths leading to a key "hobbies" in the file People.json. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.json -k "hobbies"
<p class="form-language-alternative-label">Swift</p>
try json.listPaths(filter: .key(pattern: "hobbies"))

List all the paths leading to a key starting with "h" in the file People.json. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.json -k "h.*"
<p class="form-language-alternative-label">Swift</p>
try json.listPaths(filter: .key(pattern: "h.*"))

Filter the values

The values can be filtered with one ore more predicates. When such a filter is speicified, only the single values are targeted.
A path whose value is validated by one of the provided predicates is retrieved.
A predicate will contain the variable 'value' that will be replaced to evaluate each value.
(The Swift examples use convenience initialisers).

The Swift API offers two Predicate kinds: ExpressionPredicate that takes a boolean expression and FunctionPredicate that takes a function to filter the values. Both implement the Predicate protocol. The command-line API uses ExpressionPredicate.

List the paths whose value is below 70. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.plist -v "value < 70"
<p class="form-language-alternative-label">Swift</p>
try plist.listPaths(filter: .value("value < 70"))

List the paths whose value is greater than or equal to 20 and lesser than 70. <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.plist -v "value >= 20 && value < 70"
<p class="form-language-alternative-label">Swift</p>
try plist.listPaths(filter: .value("value >= 20 && value < 70"))

List the paths whose value starts with "guit" (case sensitive). <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.json -v "value hasPrefix 'guit'"
<p class="form-language-alternative-label">Swift</p>
try plist.listPaths(filter: .value("value hasPrefix 'guit'"))

List the paths whose value starts with "guit" or are greater than 20 (case sensitive). <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.json -v "value hasPrefix 'guit'" -v "value > 20"
<p class="form-language-alternative-label">Swift</p>
try plist.listPaths(filter: .value("value hasPrefix 'guit'", "value > 20"))

To learn more about the possibilities offered by the predicates, it's possible to run scout doc -a predicates or to read this dedicated page.

Mixing up

All the features to filter the path can be mixed up.

List paths leading to Robert hobbies that contain the word "game". <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.yml "Robert.hobbies" -v "value contains 'games'"
<p class="form-language-alternative-label">Swift</p>
try yaml.listPaths(startingAt: "Robert", "hobbies", filter: .value("value contains 'games'"))

List paths leading to Robert or Tom hobbies array (group values). <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.yml "#Tom|Robert#" -k "ho.*" --group
<p class="form-language-alternative-label">Swift</p>
try yaml.listPaths(startingAt: .filter("Robert|Tom"), filter: .key(pattern: "ho.*", target: .group))

List paths leading to Suzanne's movies titles that contains the word "today". <form class="form-language"> <label> <input class="language-input input-cl" type="radio" value="cl" name="language" checked> <span class="input-label-text">Command-line</span> </label> | <label> <input class="language-input input-swift" type="radio" value="swift" name="language"> <span class="input-label-text">Swift</span> </label> </form> <p class="form-language-alternative-label">Command-line</p>

scout paths -i People.json "Suzanne.movies[:].title" -v "value contains 'today'"
<p class="form-language-alternative-label">Swift</p>
try json.listPaths(startingAt: "Suzanne", "movies", .slice(.first, .last), filter: .value("value contains 'today'"))


Miscellaneous

The code colorisation will be automatically deactivated when the output of the program is piped (thanks to an idea from Haakon Storm).


Scripting recipes

Print all the (path/single value) pairs in the data.

file="~/Desktop/Playground/People.json"
scout="/usr/local/bin/scout"

paths=(`scout paths -i $file --single`)

for path in $paths; do
	echo -n "$path: "
	$scout read -i $file $path;
done

About parsing the data

Invoking scout for each path is not efficient but gives more control. With this example, it's easy to come up with many possibilities to read or modify the data. But if one value has to be set on every path, this flexibility is too expensive to use. It will be possible in Scout 3.1.0 to use "batch" functions to run the program only once when the same value has to be set on every path. Meanwhile, it's possible to build the paths and their new values to then provide them to the program, as shown in the last recipe (Suzanne's movies new titles).

Set all "ages" key to 30 in the file People.yml

file="~/Desktop/Playground/People.yml"
scout="/usr/local/bin/scout"

paths=(`scout paths -i $file -k "age" --single`)

for path in $paths; do
	$scout set -m $file "$path=30"
done

Print the paths leading to values lesser than 30

file="~/Desktop/Playground/People.yml"
scout="/usr/local/bin/scout"

paths=(`$scout paths -i $file -k "age" --single`)

for path in $paths; do
	value=(`$scout read -i $file $path`)

	if [ $value -lt 40 ]; then
		echo "$path"
	fi
done

Add a key "language" to all Suzanne's movies with the value "fr" to the file People.plist

file="~/Desktop/Playground/People.plist"
scout="/usr/local/bin/scout"

paths=(`$scout paths -i $file "Suzanne.movies[:]" --group`)

for path in $paths; do
	$scout add -m $file "$path.language=fr"
done

Add a key "awards" with a default value if not present in Suzanne's movies array.

file="~/Desktop/Playground/People.xml"
scout="/usr/local/bin/scout"

paths=(`$scout paths -i $file "Suzanne.movies[:]" --group`)

for path in $paths; do
	if value=$($scout read -i $file "$path.awards" 2>/dev/null) ; then
		echo "'awards' key found in $path with value '$value'"
	else
		$scout add -m $file "$path.awards=No awards"
	fi
done

Set Suzanne movies's title to new ones in the file People.plist

file="~/Desktop/Playground/People.plist"
scout="/usr/local/bin/scout"

paths=(`$scout paths -i $file "Suzanne.movies[:].title"`)
newTitles=("I was tomorrow"  "I'll be yesterday" "I live in the past future")

for ((i=0; i < ${#newTitles[@]} ; i++)); do
	newTitle=${newTitles[$i+1]}
	path=${paths[$i+1]}

	$scout set -m $file "$path=$newTitle"
done

Same as above but call scout only once to set the new titles.
file="~/Desktop/Playground/People.plist"
scout="/usr/local/bin/scout"

paths=(`$scout paths -i $file "Suzanne.movies[:].title"`)
newTitles=("I was tomorrow"  "I'll be yesterday" "I live in the past future")
pathsAndNewTitles=""

for ((i=0; i < ${#newTitles[@]} ; i++)); do
	newTitle=${newTitles[$i+1]}
	path=${paths[$i+1]}
	pathsAndNewTitles="$pathsAndNewTitles '$path=$newTitle'"
done

$scout set -m $file "$path=$newTitle"