Run It Yourself

cloudarq-collect reads your AWS account on your own machine, with your own credentials, and writes what it read to one file. It sends nothing to CloudArq.

True for: cloudarq 0.3.0 and cloudarq-collect 0.3.0
Contact: [email protected]

Download the collector

One file for macOS or Linux, from the engine's v0.3.0 release, checked against the release's SHA256SUMS.

The collector's downloads
File	For
cloudarq-collect_darwin_arm64	macOS on Apple silicon
cloudarq-collect_darwin_amd64	macOS on Intel
cloudarq-collect_linux_arm64	Linux on ARM
cloudarq-collect_linux_amd64	Linux on x86-64

The first of the lines below sets asset to the file for your machine, from what uname says it is. The rest fetch it from the release, check it, and make it runnable, each step only if the one before it succeeded. The check prints the file's name and OK, and the last line prints cloudarq-collect 0.3.0.

Download and check

asset=cloudarq-collect_$(uname -s | tr '[:upper:]' '[:lower:]')_$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/')
curl -fsSLO https://github.com/CloudArq-net/cloudarq/releases/download/v0.3.0/$asset && \
curl -fsSLO https://github.com/CloudArq-net/cloudarq/releases/download/v0.3.0/SHA256SUMS && \
{ if command -v shasum >/dev/null; then shasum -a 256 -c --ignore-missing SHA256SUMS; else sha256sum -c --ignore-missing SHA256SUMS; fi; } && \
mv $asset cloudarq-collect && \
chmod +x cloudarq-collect && \
./cloudarq-collect --version

On Linux without shasum, put sha256sum -c --ignore-missing SHA256SUMS in its place, between the same &&s, so a failed check still stops the lines after it. Apple has not notarized the macOS files, so download them with curl, as above, rather than in a browser.

What it needs from AWS

Credentials for the account, allowed three calls that only read: iam:ListRoles, iam:ListAccountAliases and organizations:DescribeOrganization.

The collector takes its credentials from AWS's standard chain, as the AWS CLI does: a profile, IAM Identity Center, or the environment. It makes four calls, and only reads.

Every call it makes
Call	What it reads
sts:GetCallerIdentity	Which account the credentials belong to. AWS answers it without any permission.
iam:ListAccountAliases	The account's alias. If AWS refuses it or never answers, the account goes by its ID, the roles are still read, and the collector exits 3.
organizations:DescribeOrganization	The AWS Organization the account is in, and which account manages it. If the account is in none, the file says so, and that is no refusal. If AWS refuses the call or never answers, the file says the organization was not read, the roles are still read, and the collector exits 3.
iam:ListRoles	Every role and its trust policy, a page of up to 1,000 roles at a time.

This policy allows exactly the three calls that need a permission. Attach it to the user or role you run the collector as.

The IAM policy

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "iam:ListAccountAliases",
        "organizations:DescribeOrganization",
        "iam:ListRoles"
      ],
      "Resource": "*"
    }
  ]
}

Run it

One command. The file goes to standard output, and what the collector did goes to standard error.

Read your account

./cloudarq-collect aws --profile your-profile > estate.json

It contacts AWS alone: STS, IAM and Organizations to read your account, and whatever your AWS settings name for your credentials, such as IAM Identity Center. It sends nothing to CloudArq. --explain prints each call to AWS before it is made.

The collector's flags
Flag	What it does
--profile NAME	The AWS profile to use, as the AWS CLI's --profile takes it. Without it, the collector uses whatever AWS's standard chain finds, as the AWS CLI would.
--region REGION	The region to ask STS in, whose partition is the one asked. IAM is global, and Organizations answers from one place in each partition. Without it, the region your AWS settings name, or us-east-1.
--explain	Names each call on standard error before it is made.
--version	Used alone, as ./cloudarq-collect --version: prints the version and exits. Beside aws and its flags, the command line is not understood.

What each exit code means
Exit code	Means
0	The bundle is whole.
1	Nothing usable was written: estate.json holds no bundle; delete it rather than upload it. Standard error says why: for example, AWS did not say which account the credentials belong to.
2	The command line was not understood, and the usage is printed. estate.json holds no bundle; delete it rather than upload it.
3	A bundle, but partial: AWS refused or never answered a call, or AWS answered without what was asked, or the collector noted a limit. The bundle names what it lacks, and so does standard error.

The file it writes

estate.json is JSON in the format cloudarq.collect/v1: who the account is, the AWS Organization it is in, its roles as IAM listed them, and what AWS refused or never answered.

What estate.json holds
Field	Holds
format	Always cloudarq.collect/v1. A reader refuses any other.
provider	Always aws.
account	The account's 12-digit ID.
partition	The AWS partition, such as aws.
alias	The account's alias, or null when it has none or the call for it was refused or went unanswered.
organization	The AWS Organization the account is in, as AWS answered. Its status is member; not-a-member, when AWS answered that the account is in none; or refused, when the call was refused or went unanswered, and the refused field says why. For a member it also holds the organization's ID and ARN, its management account, that account's ARN, and its feature set. AWS's answer also carries the management account's email address, which the collector never writes. Of it, the page sends its status, the Organization's ID, the management account's ID and the feature set, as the engine reads them, and we store them; the two ARNs stay in your file.
collected_at	When the collector read the account, in UTC.
collector	The collector's name and version.
pages	Each answer iam:ListRoles gave, with its fields named as IAM names them: each role's path, name, ID, ARN, creation date, trust policy, description and maximum session duration.
refused	Each call AWS refused or never answered, or answered without what was asked: AWS's own code and message, or the code Unanswered and why: the reason the call failed, such as the network, or what AWS's answer lacked. The collector writes (removed) in each message in place of a role's session name, a user's path and name, a federated user's name, and an MFA device's or U2F key's name, and ~ in place of your home directory in the reason a call went unanswered.
limits	Each limit the collector noted, in its words.

If the collector exited 0 or 3, keep the file. Drop it on Connect in your account, where the engine reads it in your browser, and use it to check our work.