Command line tool
A core feature of the Impact Framework is the if-run
command line tool (CLI). This is how you trigger Impact Framework to execute a certain manifest file.
We also provide several other command line tools that work in concert with if-run
to enable flows such as comparing, re-executing and verifying IF output files.
This page includes reference documentation for the CLI tools, including the various commands and flags each tool exposes.
We also provide tutorial-style user documentation for these tools in the Users
section.
if-run
If you have globally installed our if
npm package, you can invoke the CLI using the if-run
command directly in your terminal. The if-run
command is an alias to npx ts-node src/index.ts
, which executes the Impact Framework's src/index.ts
script and acts as the entry point for Impact Framework.
if-run <args>
if-run
runs the full execution cycle of a manifest file, including observe
, regroup
and compute
phases along with aggregation
and explain
if they are configured in the manifest.
--manifest
, -m
The --manifest
flag is the only required flag and tells if-run
where to find the manifest file that you want to execute. This command expects to receive the path where your manifest file is saved, as shown in the following example:
if-run --manifest examples/manifests/my-manifest.yml
--output
, -o
The --output
flag is optional and is used for defining a path to save your output data.
Here is an example of --output
being used to define a path:
if-run --manifest examples/manifests/my-manifest.yml --output examples/outputs/my-outdata
## or using aliases
if-run -m examples/manifests/my-manifest.yml -o examples/outputs/my-outdata
--help
, -h
The --help
command provides information about all available commands in order to help you easily find the command you need.
Example:
if-run --help
## or using alias
if-run -h
--observe
if-run --observe
runs only the observe phase of the manifest execution. This means only those plugins that generate input
data are run. These are defined in the observe
section of the pipeline for each component in the manifest.
An example of an observe pipeline that invokes a plugin called "azure-importer" could look as follows:
tree:
children:
child:
pipeline:
observe:
- azure-importer
--regroup
if-run --regroup
runs only the regrouping phase of the manifest's execution. There has to be input
data available in the manifest to regroup (or --observe
has to be invoked too) and the regrouping configuration has to be included in the manifest. This config defines which parameters if-run --regroup
should regroup the data by.
For example, to regroup on cloud/region
and cloud/instance-type
:
tree:
children:
child:
pipeline:
observe:
regroup:
- cloud/region
- cloud/instance-type
--compute
if-run --compute
runs only the compute phase of the manifest's execution. The manifest passed to if-run --compute
should already have input data, appropriately grouped (or you have to pass --observe --regroup
too). This includes the plugins that do operations over the input data to generate output data.
For example, in a manifest that executes sum
, coefficient
and multiply
in its compute phase:
tree:
children:
child-1:
pipeline:
observe:
compute:
- sum
- coefficient
- multiply
--debug
You can provide the --debug
flag to if-run
in order to display execution logs to the console. These logs show messages for each operation IF and its plugins are executing. For example, your debug
logs will look similar to the following:
INFO: 2024-06-12T08:48:02.918Z: Starting IF
DEBUG: 2024-06-12T08:48:02.919Z: Loading manifest
DEBUG: 2024-06-12T08:48:02.924Z: Capturing runtime environment data
DEBUG: 2024-06-12T08:48:03.978Z: Validating manifest
DEBUG: 2024-06-12T08:48:03.980Z: Syncing parameters
DEBUG: 2024-06-12T08:48:03.980Z: Initializing plugins
DEBUG: 2024-06-12T08:48:03.981Z: Initializing Sum
DEBUG: 2024-06-12T08:48:03.981Z: Loading Sum from builtin
DEBUG: 2024-06-12T08:48:04.859Z: Initializing Coefficient
DEBUG: 2024-06-12T08:48:04.859Z: Loading Coefficient from builtin
DEBUG: 2024-06-12T08:48:04.860Z: Initializing Multiply
DEBUG: 2024-06-12T08:48:04.860Z: Loading Multiply from builtin
DEBUG: 2024-06-12T08:48:04.860Z: Computing pipeline for `sum`
DEBUG: 2024-06-12T08:48:04.861Z: Computing pipeline for `coefficient`
DEBUG: 2024-06-12T08:48:04.861Z: Computing pipeline for `multiply`
DEBUG: 2024-06-12T08:48:04.862Z: Aggregating outputs
DEBUG: 2024-06-12T08:48:04.862Z: Preparing output data
You can use the --debug
flag to help debug failing IF runs. You will see exactly where in the execution pipeline an error arose. If the error arose from a plugin, this will be clear from the execution logs, for example:
INFO: 2024-06-12T08:53:21.376Z: Starting IF
DEBUG: 2024-06-12T08:53:21.376Z: Loading manifest
DEBUG: 2024-06-12T08:53:21.381Z: Capturing runtime environment data
DEBUG: 2024-06-12T08:53:22.367Z: Validating manifest
DEBUG: 2024-06-12T08:53:22.369Z: Syncing parameters
DEBUG: 2024-06-12T08:53:22.369Z: Initializing plugins
DEBUG: 2024-06-12T08:53:22.369Z: Initializing Sum
DEBUG: 2024-06-12T08:53:22.370Z: Loading Sum from builtin
DEBUG: 2024-06-12T08:53:23.165Z: Initializing Coefficient
DEBUG: 2024-06-12T08:53:23.165Z: Loading Coefficient from builtin
DEBUG: 2024-06-12T08:53:23.165Z: Initializing Multiply
DEBUG: 2024-06-12T08:53:23.165Z: Loading Multiply from builtin
DEBUG: 2024-06-12T08:53:23.165Z: Computing pipeline for `sum`
[2024-06-12 09:53:23.166 AM] error: cpu/energy is missing from the input array.
if-diff
The if-diff
command line tool allows you to determine whether two manifest or output files are the same, and if not, how they differ.
if-diff
needs two files to compare - a source
and a target
. The source
file is considered to be the "true" file that another file, the target
, is compared against. Note that for most purposes, it doesn't matter which file is assigned as source
or target
- the important thing is that if-diff
receives two files. Both files should be yaml
files. They are expected to be IF output files, meaning they contain all the required fields of a manifest
plus the IF-generated output
and execution
blocks.
if-diff
is run as follows:
if-diff --source file-1.yml --target file2.yml
You can also pipe the outputs from if-run
directly into if-diff
. This means you only provide one file to if-diff
and the other comes from a new if-run
run configured to send its output data to the console. This is an important feature because it allows you to receive an output file and verify that it was computed correctly and not tampered with post-execution. For example, if someone provides you with an output file, you can strip out the outputs
section and re-run it with if-run
, piping the outputs straight to if-diff
to compare against the original you received.
If the original was correctly and honestly reported, if-diff
will return a success response.
e.g.
if-run -m my-manifest | if-diff --target my-output-file.yml
if-diff
matching rules
if-diff
looks for differences between the source
and target
. However, if-diff
applies its own IF-specific matching rules, ensuring that the outputs are functionally identical even if they are not precisely identical. For example, if-diff
allows the order of nodes in a tree to vary between files as long as identically named components contain identical data.
Difference identified | Report or ignore? | Note |
---|---|---|
trees contain different number of nodes | report | works the same regardless whether source or target has more nodes |
nodes in tree have different names | report | There should be no named nodes existing in one file that aren't also in the other |
nodes in tree contain non-identical fields and/or values | report | the data inside each tree component should contain identical keys/values |
keys and values in context field are non- identical | report | the same fields should exist in the context section and their values should be identical |
status and error fields in execution block | report | Only these two fields in execution are considered |
order of nodes in tree are different | ignore | if data is identical, position of node in tree is ignored |
order of fields in context | ignore | if data is identical, position of field in context is ignored |
content of execution block EXCEPT status and error | ignore | environment information is ignored |
if-diff
outputs
If if-diff
finds no in-scope differences between the source
and target
then it returns a success message and exit code 0
:
FILES MATCH and exit code 0.
If if-diff
detects an in-scope difference between the files, it halts execution, returns exit code 1
and reports the difference to the command line.
The report includes the yaml path to the differing element in the tree, the value in the source
and the value in the target
, using the following schema:
Files do not match!
<yaml path to non-matching element>
source: <value in source file>
target: <value in target file>
If the difference relates to a missing node in the tree for source or target then <value in x file>
should be either exists or missing and the yaml path should point to the highest level element that is missing (e.g. if an entire child component is missing, provide the path to the child component).
e.g. different values detected for a given key in an input array:
Files do not match!
tree.children.vm1[4].cpu/utilization
source: 45
target: 43
e.g. different components in tree
in source
and target
:
Files do not match!
tree.children.child1
source: missing
target: exists
if-env
if-env
is a command line tool that helps you to create local development environments where you can run manifests.
There are two use cases for this:
- setting up a new development environment for plugin building
- replicating a runtime environment for a given manifest, so you can re-execute it
commands
--manifest
or-m
: the path to a manifest whose dependencies you want to install--install
or-i
: instructsif-env
to automatically install the dependencies in the localpackage.json
--cwd
or-c
: forcesif-env
to create or update the package.json in the current working directory. This is already default behaviour when no arguments are passed toif-env
, but when a manifest is passed to-m
,if-env
defaults to saving a package.json in the same folder as the manifest. using-cwd
overrides that behaviour and uses the current working directory as thepackage.json
target path.
Setting up new development environments using if-env
If you are creating a new manifest from scratch and want to bootstrap your way in, you can use if-env
with no arguments to generate a template manifest and package.json in your current working directory. Then, all you need to do is tweak the templates for your specific use case.
For example:
mkdir my-manifest && cd my-manifest
if-env
After running these commands, you will see the following files in my-manifest
:
ls my-manifest
> package.json manifest.yaml
Now, you can use these files as templates for your manifest development.
Replicating runtime environments using if-env
If you are given an IF output file and you want to re-run it, you can use if-env
to install that output file's dependencies so that all the plugins in its execution pipeline can be executed.
For example, if you are given a file, output-file.yml
, you can save the file to if
and run
cd if
if-env -m output-file.yml
if-env
will compare the installed dependencies in the package.json
it sees in if
with the dependencies listed in output-file.yaml
. Any dependencies that are in output-file.yaml
and not in if/package.json
will be added to if-package.json
. Then, you can run:
npm i
and you are ready to re-execute output-file.yaml
in your local environment. We also provide the --install
flag to instruct if-env
to automatically run npm i
after merging the dependencies, so you could craft a single command to install all the relevant dependencies and then run the manifest, as follows:
if-env -m output-file.yml -i && if-run -m output-file.yml
if-check
if-check
is a manifest verification tool that is equivalent to running if-env
and if-diff
on a given manifest file. The manifest file must have outputs
and an execution
section for if-check
to run.
The intended use case is to verify that a manifest's outputs are correct and honest. Say someone handed you a manifest as evidence of their environmental impact. You could choose to trust them, or you could run if-check
to verify that their calculations are correct. Under the hood, IF is creating a development environment using the dependencies listed in the given file's execution
section and then executing the file locally, then comparing the newly generated results to those in the given file.
To check a file:
if-check -m <path-to-file>
If the if-check
is successful you will receive the following response:
if-check: successfully verified <filename>
If if-check
was not able to verify the file because there were differences in the given and re-executed files, then you will receive the following response which includes the details of how the files differ, as per if-diff
.
if-check: could not verify <filename>. The re-executed file does not match the original.
Running IF over multiple manifests with --d
Alice could also run if-check
over any number of manifests in a single command, using the --directory
or -d
subcommand. For a folder containing multiple manifests, pass the folder path:
if-check -d /my-folder-of-manifests
Each manifest will be run through if-check
in sequence.
if-csv
if-csv
is a command line tool that helps to save data to CSV file.
commands
--manifest
or-m
: (optional) the path to an executed manifest--output
or-o
: (optional) the path to save your output data incsv
format--params
or-p
: (required) the metric to export the data
There are three use cases for this:
- Exporting CSV with the
--output
flag: When the--output
flag is provided,if-csv
exports the data to a CSV file at the specified path. This is useful for saving data for later use or sharing with others.
if-csv -m ./my-manifest.yml -p carbon -o ./my-outdata
- Printing CSV to the console without the
--output
flag: If the--output
flag is omitted,if-csv
will print the CSV data directly to the console. This is useful for quick checks.
if-csv -m ./my-manifest.yml -p carbon
- Piping output from
if-run
toif-csv
. By piping the output fromif-run
, you can chain commands to execute a manifest and then immediately export the data to a CSV file.
if-run -m ./my-manifest.yml | if-csv -p carbon -o ./my-outdata