Thursday, August 11, 2022
HomeCloud ComputingDo your AWS deployments match your Terraform definitions? Use SQL to seek...

Do your AWS deployments match your Terraform definitions? Use SQL to seek out out.


In “How SQL can unify entry to APIs” I made the case for SQL as a standard surroundings during which to motive about information flowing from many alternative APIs. The important thing enabler of that situation is Steampipe, a Postgres-based software with a rising suite of API plugins that map APIs to overseas tables in Postgres.

These APIs had been, initially, those supplied by AWS, Azure, and GCP. Such APIs are sometimes made extra accessible to builders by means of wrappers like boto3. A typical SQL interface is arguably a greater unifier of the sprawling API ecosystems inside these clouds, and that’s inarguably true in multicloud eventualities. With Postgres below the hood, by the best way, you’re not restricted to SQL: You possibly can hook Python or JavaScript or one other language to Postgres and leverage the frequent SQL interface from these languages too.

The Steampipe ecosystem then expanded with plugins for a lot of different companies together with GitHub, Google Workspace, IMAP, Jira, LDAP, Shodan, Slack, Stripe, and Zendesk. Becoming a member of throughout these APIs is a superpower finest confirmed by this instance that joins Amazon EC2 endpoints with Shodan vulnerabilities in simply 10 strains of very fundamental SQL.

  aws_ec2_instance a
left be a part of
  shodan_host s on a.public_ip_address = s.ip
the place
  a.public_ip_address is just not null;

| instance_id         | ports    | vulns              |
| i-0dc60dd191cb84239 | null     | null               |
| i-042a51a815773780d | [80,22]  | null               |
| i-00cf426db9b8a58b6 | [22]     | null               |
| i-0e97f373db42dfa3f | [22,111] | ["CVE-2018-15919"] |

Information are APIs too

However what’s an API, actually? Should it at all times entail HTTP requests to service endpoints? Extra broadly APIs are information sources that are available in different flavors too. Net pages are sometimes, nonetheless, de facto APIs. I’ve accomplished extra net scraping than I care to consider over time and the ability stays helpful.

Information are additionally information sources: configuration recordsdata (INI, YAML, JSON), infrastructure-as-code recordsdata (Terraform, CloudFormation), information recordsdata (CSV). When plugins for these sources started to hitch the combination, Steampipe grew to become much more highly effective.

First got here the CSV plugin, which unlocked all kinds of helpful queries. Think about, for instance, how we frequently faux spreadsheets are databases. In doing so we are able to assume there’s referential integrity when actually there isn’t. Should you export spreadsheet information to CSV, you should use SQL to discover these flawed assumptions. And that’s simply one of many infinite methods I can think about utilizing SQL to question the world’s main file format for information change.

Then got here the Terraform plugin, which queries Terraform recordsdata to ask and reply questions like: “Which trails are usually not encrypted?”

the place
  and arguments -> 'kms_key_id' is null;

Utilizing the AWS plugin’s aws_cloudtrail_trail desk, we are able to ask and reply the identical query for deployed infrastructure, and return a consequence set that you may UNION with the primary one.

  arn as path
the place
  kms_key_id is null;

Ideally the solutions will at all times be the identical. What you stated must be deployed, utilizing Terraform, ought to match what’s really deployed for those who question AWS APIs. In the true world, after all, upkeep and/or incident response can lead to configuration drift. Given a standard solution to motive over outlined and deployed infrastructure, we are able to handle such drift programmatically.

Belt and suspenders

For deployed infrastucture, Steampipe has lengthy supplied a set of mods that layer safety and compliance checks onto API-derived overseas tables. The AWS Compliance mod, for instance, gives benchmarks and controls to verify deployed infrastructure towards eleven requirements and frameworks together with CIS, GDPR, HIPAA, NIST 800-53, and SOC 2.

steampipe aws cis v140 console IDG

With the arrival of the Terraform plugin it grew to become doable to create complementary mods, like Terraform AWS Compliance, that present the identical sorts of checks for outlined infrastructure.

steampipe terraform aws compliance console output IDG

Does what you outlined final month match what you deployed yesterday? A passable reply requires the flexibility to motive over outlined and deployed infrastructure in a standard and frictionless manner. SQL can’t take away all of the friction however it’s a strong solvent.

Copyright © 2022 IDG Communications, Inc.



Most Popular

Recent Comments