Taking configuration transformation and separation a step further

In a previous post I’d defined a simple VPC peering based network solution. There I highlighted how I prefer to separate declarative configuration from logic; defining the data in terms of how people interact with it on a day-to-day basis rather than how it’s used in code to interact with machines.

In that post I’d mentioned how one could imagine defining abstract interfaces to handle the idiosyncrasies between vendors. If you haven’t read that, feel free to skip over my network musings, but at least go back and familiarize yourself with the configuration format.

If you want to see the source for this you can find it here¹

Since we’re already transforming our configuration to a more automation-friendly form there is no reason we can’t abstract the vendor specific details away. The interface for subnets and their definition could look like so.

export interface SubnetData {}
export type SubnetDefinition<T extends SubnetData> = PartialRecord<
  Subnets,
  // Subnets are definied in our config.  This enforces valid names
  T
>;
export interface NetworkData<T> {
  subnets: SubnetDefinition<T>;
}

export type NetworkConfig<T, N extends NetworkData<T>> = Record<
  Vpcs,
  // Vpcs are definied in our config.  This enforces valid names
  N
>;

Given this, we can extend the interfaces; ensuring that our datastructures between cloud vendors are identical while sweeping the vendor specific configuration under the rug.

Config transformation examples

As we did previously, we’ll take a configuration that I like to work with directly and transform it to something easier to program with. We’ll add/transform to the vendor specific options as we go.

GCP specific interfaces

interface S extends SubnetData {
  config: {
    project: string;
    ipCidrRange: string;
    gatewayAddress: string;
  };
}
interface N<S> extends NetworkData<S> {
  project: string;
}

export class GcpNetworkingConfig implements NetworkDefinition {
  topo = topo;
  peers = peers;
  firewall = firewall;
  config: NetworkConfig<S, N<S>>;

  constructor(project: Project) {
    const l = getKeys(this.topo).map((j) => {
      const subnets = this.topo[j].subNets;
      return {
        [j]: {
          project: project.name,
          subnets: getKeys(subnets)
            .map((k) => {
              return {
                [k]: {
                  config: {
                    project: project.name,
                    ipCidrRange: subnets[k].cidr,
                    gatewayAddress: subnets[k].gateway,
                  },
                },
              } as SubnetDefinition<S>;
            })
            .reduce((obj, i) => {
              return { ...obj, ...i };
            }),
        },
      } as NetworkConfig<S, N<S>>;
    });
    this.config = l.reduce((obj, i) => {
      return { ...obj, ...i };
    });
  }
}

AWS

interface S extends SubnetData {
  config: {
    cidrBlock: string;
  };
}

interface N<S> extends NetworkData<S> {
  cidrBlock: string;
}

export class AwsNetworkingConfig implements NetworkDefinition {
  topo = topo;
  peers = peers;
  firewall = firewall;
  config: NetworkConfig<S, N<S>>;

  constructor() {
    const l = getKeys(this.topo).map((j) => {
      const subnets = this.topo[j].subNets;
      return {
        [j]: {
          cidrBlock: this.topo[j].cidr,
          subnets: getKeys(subnets)
            .map((k) => {
              return {
                [k]: {
                  config: {
                    cidrBlock: subnets[k].cidr,
                  },
                },
              } as SubnetDefinition<S>;
            })
            .reduce((obj, i) => {
              return { ...obj, ...i };
            }),
        },
      } as NetworkConfig<S, N<S>>;
    });
    this.config = l.reduce((obj, i) => {
      return { ...obj, ...i };
    });
  }
}

The implementation portion

This still leaves a fair amount of duplication/boilerplate between the implementations as you can see.

examples

AWS

export class AwsNetwork extends Construct {
  readonly config: NetworkingConfig;
  readonly vpcs: Record<Vpcs, vpc.Vpc>;
  readonly subnets: Record<Vpcs, Record<Subnets, vpc.Subnet>>;
  readonly peers: PartialRecord<
    Vpcs,
    PartialRecord<Vpcs, vpc.VpcPeeringConnection>
  >;
  readonly firewalls: Record<
    Vpcs,
    Record<Direction, Record<string, vpc.NetworkAclRule[]>>
  >;

  constructor(
    scope: Construct,
    name: string,
    config: NetworkingConfig,
  ) {
    super(scope, name);
    this.config = config;
    this.vpcs = this.networks(config);
    this.subnets = this.subnetworks(config);
    this.peers = this.networkspeering(config);
    this.firewalls = this.firewall(config);
  }
[snip]

GCP

export class GcpNetwork extends Construct {
  readonly config: NetworkingConfig;
  readonly vpcs: Record<Vpcs, ComputeNetwork>;
  readonly subnets: Record<Vpcs, Record<Subnets, ComputeSubnetwork>>;
  readonly peers: PartialRecord<
    Vpcs,
    PartialRecord<Vpcs, ComputeNetworkPeering>
  >;
  readonly firewalls: Record<
    Vpcs,
    Record<Direction, Record<string, ComputeFirewall>>
  >;
  constructor(
    scope: Construct,
    name: string,
    config: NetworkingConfig,
  ) {
    super(scope, name);
    this.config = config;
    this.vpcs = this.networks(config);
    this.subnets = this.subnetworks(config);
    this.peers = this.networkspeering(config);
    this.firewalls = this.firewall(config);
  }
[snip]

We could implement a class that takes an abstract type instead of the actual constructs like vpc.Subnets, ComputeSubnet, etc. This seems like it would be doable but would take at least

1. The class itself would have to accept a function from the GCP/AWS implementation
2. modifications to the type that describes our transformed configuration
3. modifications to the constructors that generate the config

But at that point abstracting may cost more than some boilerplate. I’ll think about it some more. If you have thoughts I’d love to hear them.