Introduction​

Welcome to an update on go-c8y-cli. For those who are new to the scene, go-c8y-cli is a cli tool used to interact with the powerful and very stable Cumulocity IoT API. It is developed to make you as productive as possible so that you can get the most out of your IoT Platform with minimal effort whilst protecting you against stupid mistakes.

Development of it started a few years ago when I was working for a company which used the Cumulocity IoT platform. I quickly ran into limitations with the defacto PostMan and also wanted a solution which enabled me to quickly explore/fix/create items in Cumulocity IoT and be able to re-use the same commands in a script to automate a process. I made it open source as I found that if it can make my life easier, then other people should also be able to benefit, and it grew from there.

Now we have an open source tool which is adaptable to many use-cases and tries to encourage best practices, whilst be super composable so it can be used for both basic and more complex scenarios.

As you might guess, it is developed in golang and it is a single compiled binary available on a large cross section of platforms.

Common use-cases​

Below show some of the common use-cases built from a combination of some of my daily tasks, and some feedback from real users:

• Tenant setup (create tenants, add users, configure retention rules)
• API discovery
• Troubleshooting - Something went wrong, but when/how/why?
• Simulation / Rapid Prototyping
• CI/CD - deployment, cleanup
  • Upload and activate web applications and microservices
  • Remove device certificates
  • Delete devices after tests have run
• Reporting

What's New​

go-c8y-cli is constantly being improved and new features are added frequently. Some of the goodies that have been developed over the last year include:

• Support for Extensions - easier to create/share/install extensions
• Progress bars for binary uploads and downloads
• Support for notifications2 API
• Output templates - shape the output of your data using Jsonnet templates
• Additional template (jsonnet) functions (e.g. for date manipulation, get/select properties using dot notation)
• csv utilities
• DataHub API

Installation​

As always a tool is useless unless it can be installed on your system.

• shell
• Docker
• PowerShell

Or you can always download the binary directly using some new instructions (which will eventually also make it on the website as well)

• Manually installing go-c8y-cli

Common misconceptions​

Just to clarify a few common misconceptions which I've heard in the last year or two:

• It relies on PowerShell...No, this has not been the case since June 2021!
• This is just postman on the command line right?...nope, it does so much more!

Basics​

Getting around​

Now for the demo part, the following items will be quickly shown so that it will help you navigate go-c8y-cli and understand the basic principles.

• Creating / activating a session
  • Caveats: SSO users are not supported due to a platform limitation (though this may change in the future)
  • Protecting against dumb mistakes
• Using command completion
• Help and examples
• Viewing data and output formats
• Piping between commands
• Template generation
• Online documentation

Use-cases

This section shows some examples of real world use-cases which I have personally needed at some point in time, or other people have are using in their Cumulocity IoT journey.

Exploring your data​

List of device types used in the platform​

Sometimes it is helpful to view get an overview of the different device types in your tenant. The go-c8y-cli can be easily combined with other commands to extract the information that you need:

c8y devices list --includeAll --select type -o csv | sort | uniq -c | sort --human-numeric-sort

Cleaning up devices​

Use case: The dev team is prototyping a lot and slowly there are more and more devices in the platform which are no longer being used.

c8y devices list --type Linux | c8y devices delete -f

You can also run use multiple workers to process the input which will make the overall job quicker, however you should always do this in moderation, as you don't want to put unnecessary load on the platform for an admin task.

c8y devices list --type nx_Microservice --creationTimeDateTo -60d --pageSize 100 \
| c8y devices delete --progress --workers 5 --delay 1000ms

Exploring the api​

go-c8y-cli is also a great tool for API discovery, and also converting the commands to actual HTTP requests which can be ported to other languages, e.g. Python. There is a bit of overlap with the interactive OpenAPI spec, however go-c8y-cli tries to put all of the commands in the context of the user, rather than a pure API view. For example c8y devices list is just doing a managed object query with a specific filter on top, and there is no specific "device" endpoint.

Use commands to view the query in your expected context​

You can inspect what the actual HTTP call looks like without sending it to Cumulocity IoT. This helps you better understand the structure of the API, and you can also port the HTTP request to what ever language you want, and you have the exact call that you need.

c8y devices list --name "tedge_*" --creationTimeDateFrom -10d --dry

What If: Sending [GET] request to [https://test-ci-runner01.latest.stage.c8y.io/inventory/managedObjects?q=%24filter%3D%28name+eq+%27testme%27%29+and+%28creationTime.date+ge+%272023-08-27T10%3A16%3A05.06709%2B10%3A00%27%29+%24orderby%3Dname]

### GET /inventory/managedObjects?q=$filter=(name eq 'testme') and (creationTime.date ge '2023-08-27T10:16:05.06709+10:00') $orderby=name

| header            | value
|-------------------|---------------------------
| Accept            | application/json
| Authorization     | Bearer {token}

Troubleshooting​

Troubleshooting is a common task throughout your IoT journey. It can range from a root cause analysis to respond to a production event, or simply checking which devices don't conform to the expected data schema.

Check which devices don't have an external identity​

In a perfect world, every device will have an external identity (which makes it easier and quicker to reference). However during development it is fairly easy to forget things, or there is a bug which causes the creation of the external identity to fail.

The following one-liner shows how to check which devices

c8y devices list -p 10 \
| c8y identity list --outputTemplate "if std.length(output) == 0 then input.value else ''" --select id,name,type,self -o json

Though it is always good to narrow down your search space first before doing this as checking the identity requires one request per device, so in reality you probably want to filter the devices first, and then check if the identity exists or not:

c8y devices list --query "has(myFragment) and name eq 'test*'" \
| c8y identity list --outputTemplate "if std.length(output) == 0 then input.value else ''" --select id,name,type,self -o json

Viewing the inventory data structure​

Being able to inspect the managed object information (e.g. devices, agents etc.) can be really useful to spot why something isn't working or just to spot inconsistencies across devices (which happens when you're working in larger team).

You can inspect single items and pretty print it using some of the default aliases (to reduce the amount of typing for common tasks):

c8y mo 43903

Downloading multiple binaries files from the platform​

The larger your fleet of devices, the harder it can to do some root cause analysis on them. Using the power of pipelines it makes a manual task such as downloading event binaries (aka attachments) a simple one-liner:

c8y events list --type c8y-configuration-plugin \
| c8y events downloadBinary --noProgress

Or using the power of output templates, you can even add combine information in the request with the output:

c8y events list --type c8y-configuration-plugin \
| c8y events downloadBinary --noProgress --outputTemplate "'\n\n# Device: ' + request.url + '\n' + output + '\n'" \
| sed 's|\\n|\n|g'

Saving the files to disk

c8y events list --type c8y-configuration-plugin \
| c8y events downloadBinary --noProgress --outputFileRaw "{filename}.txt"

Download device logs​

This is a more advanced example but it is meant to just show how useful the pipeline is to chain multiple commands together to build a complex workflow.

echo 301 \
| c8y bulkoperations listOperations --status FAILED --includeAll \
| c8y operations create --template "{c8y_LogfileRequest:{logFile:'software-management',dateFrom: _.Date(input.value.creationTime, '-120min'), dateTo: _.Date('now'), maximumLines:1000,searchText:''}}" --description "Collect logs" -f \
| c8y operations wait --status SUCCESSFUL --select c8y_LogfileRequest.file -o csv | c8y api

The above command is doing the following:

1. Get a list of all of operations which failed related to a specific bulk operation
2. Create an operation on the device where the operation failed to collect a specific log file using a time filter 2 hours before the creation of the operation until now.
3. Wait until the log file request operation is finished, then pipe the link to the file
4. Download the file (using the generic c8y api command)

This allows you to write all of the information to a file where you can use grep to see what was happening

CI/CD​

A new GitHub Actions called setup-go-c8y-cli has been added to make it even easier to install go-c8y-cli in your workflow.

Check out the example workflow for some examples of what is possible.

Deploying applications​

Deploying microservices and web applications are simple one-liners which are able to deploy new versions of your applications directly in your workflow.

go-c8y-cli can also be used to clean up old versions of web applications so that your platform makes the most efficient use of resources.

1. Deploy a microservices
2. Deploy a web application
3. Delete older versions of the web application only keeping the last X versions

Device Management​

go-c8y-cli is not just limited to deploying applications, but you can also use as part of a GitOps style device manager which looks at deploying software updates to devices.

The device management part of the workflow involves

1. Create/Upload a new configuration binary to the Configuration Repository
2. Create/Upload a new version of software to the Software Repository
3. Install software on a small group of devices (e.g. Canary group) via a Bulk Operation
4. Check if installation was successful
5. If the previous step finished with a pass rate of > X% then install the same software on the remaining devices