Skip to content

JQ.md

Latest commit

 

History

History
102 lines (84 loc) · 4.11 KB

JQ.md

File metadata and controls

102 lines (84 loc) · 4.11 KB

JQ Tips & Tricks

# Array Filter & Map
aws organizations list-organizational-units-for-parent --parent-id r-y70b | jq  '.OrganizationalUnits | {OrganizationalUnits: map({Id: .Id, Name: .Name})}'

Filter dict by Keys

# Dict Filter keys for {}
# Returns IAM Account summary for Groups, Policies, Roles & Users plus Qoata
aws iam get-account-summary | jq --sort-keys '.SummaryMap | {SummaryMap: {Groups: .Groups, GroupsQuota: .GroupsQuota, Policies: .Policies, PoliciesQuota: .PoliciesQuota, PolicyVersionsInUse: .PolicyVersionsInUse, Roles: .Roles, RolesQuota: .RolesQuota, Users: .Users,UsersQuota: .UsersQuota}}'

Filter list of dict by keys

cat data/cloud-hub-90-days-signin-2023-10-06.json | jq '.Records | {Records: map({eventTime: .eventTime, recipientAccountId: .recipientAccountId, userIdentity: .userIdentity, eventCategory: .eventCategory, eventName: .eventName, eventType: .eventType, eventSource: .eventSource, responseElements: .responseElements})}'

Filter list of dict for one key

aws directconnect describe-direct-connect-gateways | jq '.directConnectGateways | map(.directConnectGatewayId)' -r  -r | grep -E -v '\[|\]' | tr -d ',"' 

ids=$( aws directconnect describe-direct-connect-gateways | jq '.directConnectGateways | map(.directConnectGatewayId)' -r  -r | grep -E -v '\[|\]' | tr -d ',"' )

for id in $ids; do
  aws directconnect describe-direct-connect-gateways $id --output yaml
done

Filter list of dicts by keys & values

Json data mocel:
``

{
  "Accounts": [
    {
        "Id": "883654865448",
        "Arn": "arn:aws:organizations::171844140004:account/o-jmt3aajwbh/883654865448",
        "Email": "aws.gov-svc-acct-cc001-commercial@ge.com",
        "Name": "gov-svc-acct-cc001-commercial",
        "Status": "ACTIVE",
        "JoinedMethod": "CREATED",
        "JoinedTimestamp": "2023-07-05T08:43:51.166000-07:00"
        }
  ]
}

SCP Name & ID

aws organizations list-policies --filter SERVICE_CONTROL_POLICY | jq 'reduce .Policies[] as $item ({}; .[$item.Name] = $item.Id)' | yq . -y

VPC Flow Logs

AWS_PROFILE=Public-Cloud-Dev aws ec2 describe-flow-logs | jq '.FlowLogs | map ({CreationTime: .CreationTime, DeliverLogsStatus: .DeliverLogsStatus, FlowLogStatus: .FlowLogStatus, ResourceId: .ResourceId, TrafficType: .TrafficType, LogDestinationType: .LogDestinationType, LogDestination: .LogDestination, MaxAggregationInterval: .MaxAggregationInterval})'

JQ Query

# Filter for Active
aws organizations list-accounts | jq '.Accounts | map(select(.Status == "ACTIVE" )) | {Accounts: map({Id: .Id, Name: .Name, Status: .Status})}'


aws organizations list-accounts | jq '.Accounts | {Accounts: map({Id: .Id, Name: .Name, Status: .Status})}'

# Converting to csv
aws organizations list-accounts | jq '.Accounts[] | [.Id, .Name, .Status] | @csv' -r

# Add Cutom key/val to dictionary
REGION='us-east-1'
REGION_NAME='US East (Ohio)'
aws guardduty list-invitations --region $REGION | jq --arg "Region" "$REGION" --arg "RegionName" "$REGION_NAME" --arg "Service" "GuardDuty" '.Invitations[] + $ARGS.named'

# Lenght & Filter$
EXP_DATE="$(date -d '180 days ago' --iso-8601=seconds)"  # 180 days ago, # [[ $EXP_DATE > $CHK_DATE ]] && echo 'Expired' || echo 'Active' 
REGION='us-east-1'
JQ='.Snapshots | map({SnapshotId: .SnapshotId, StartTime: .StartTime})'
aws ec2 describe-snapshots --owner-ids self --region $REGION | jq "$JQ"
aws ec2 describe-snapshots --owner-ids self --region $REGION | jq "$JQ | length"

# Print array as text
echo '[ "apple", "banana", "cherry" ]' | jq -c '.[]' -r
echo '[ "apple", "banana", "cherry" ]' | jq -cr '.[]'

# Get text of list of dicts with multiple field join
# Returns text of Path + RoleNmae of Roles: [{},{},...]
aws iam list-roles | jq -r '.Roles[] | "\(.Path)\(.RoleName)"'

JQ Filter

# Filter for .business_unit == "Aviation" and .vendor == "AWS"
cat ../data-export/cmc-data.json | jq '.[] | select(.business_unit == "Aviation" and .vendor == "AWS")'

YQ

# Minify JASON of YAML File
yq . filename | jq . -c

# Minify JASON Length of YAML File
yq . filename | jq . -c | wc | awk '{print $3}'