0.18.0

Lots of good stuff in this release. Highlights include:

• Logging for c2c iptables can be enabled through a BOSH property
• Container networking scales to 20K application instances with 3 policies per application.
• Initial support for logging ASG iptables through a BOSH property. ASG logs will be prefixed with OK_ or DENY_.
• If you are running Diego release v1.10.1 you must upgrade to this release

We do not recommend using cf-networking-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com.

Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

New Manifest Properties

• cf_networking.rep_listen_addr_admin enables our drain scripts to wait for the Diego rep to exit.
  It should always be the same value as diego.rep.listen_addr_admin. It defaults to 127.0.0.1:1800.
• cf_networking.garden_external_networker.iptables_asg_logging globally enables iptables logging for
  all ASGs, including logging of denied packets. Defaults to false.
• cf_networking.vxlan_policy_agent.iptables_c2c_logging enables iptables logging for
  container-to-container traffic. It defaults to false. Note: this is already
  configurable at runtime.
• cf_networking.plugin.health_check_port allows BOSH to better health-check the flanneld process
  required for connectivity.

Removed Manifest Properties

• cf_networking.policy_server.database.connection_string was deprecated in v0.10.0 and is now removed.

Significant Changes

Scalability

• container networking is reliable with 20k app instances across 100 diego cells
• Scalability test for popular server
• Our docs include recommendations on scaling policy server instances and DB
• The policy server can handle our scalability target of 20K AIs

Upgrades

• As an operator I can upgrade from kawasaki to cf-networking

Manifest Changes

• Remove deprecated BOSH property policy_server.database.connection_string

Security

• Policy server can communicate with UAA over TLS

Chores

• Investigate and fix "Ginkgo timed out waiting for parallel nodes to report back"
• Improve stop behavior of monit ctl scripts

Stability

• Flannel has a healthcheck endpoint for monit
• A cell with a subnet mismatch can be recovered by a BOSH restart of the cell
• Policy server monit script checks a healthcheck endpoint

Logging

• Logging for c2c iptables is configurable through a BOSH property
• Logging for denied outbound non-c2c packets

Internal integration

• Garden External Networker handles NetIn/NetOut fields during Up

0.17.0

This release reduces the Flannel subnet lease renewal interval to alleviate the effects of etcd failures. It also includes a manifest change. Take a look at the manifest change log for details.

We do not recommend using cf-networking-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com.

Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

Significant Changes

Flannel

• Document risks and mitigations for container networking when etcd disappears and comes back with an empty data dir
• Flannel subnet range for a cell should be configurable

Scalability

• Policy server flakes when trying to add/delete several thousand policies
• As an operator I have metrics to help evaluate policy server performance
• As a space developer, I expect list policies to work when there are a lot of policies/apps
• Our docs include recommendations on scaling policy server instances and DB
• When policies are requested by ID, policy server does not query database for all policies

Chores

• Migrate pickelhelm to GCP
• Integrate CNI master and provide feedback to the CNI maintainers

0.16.0

No big manifest changes in this release - key changes include a property to override the interface MTU, policy cleanup for deleted applications and spaces and CLI enhancements.

We do not recommend using cf-networking-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com. Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

Significant Changes

Manifest Changes

• MTU for containers is configurable

Policy Cleanup

• Policy server cleanup works when I have 10000 apps but only few configured with policy

CLI

• Update CATS and runtime-ci docker image to use remove-access
• As a space developer I get a meaningful error message when I don't have network.write scope and try to configure a policy

Security

• As an attacker I can send a very large GUID in my call to the policy server API

Documentation

• Our docs include information about the policy server DB

Metrics

• As an operator I can set up an alert for when my cell has a Flannel watchdog error
• Investigate increased policy server response time on toque
• As an operator I have metrics to help evaluate policy server performance

Miscellaneous

• Build a demo to show UDP container networking
• Move httpClient test coverage from policy-server/cc_client to lib/json_client, and refactor

0.15.0

This release includes significant manifest changes. Please take a look at the manifest changelog for details.

We do not recommend using cf-networking-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com. Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

Significant Changes

Manifest Changes

• Simplify / consolidate BOSH properties
• Rename netman to cf-networking for all container networking artifacts

Policy Cleanup

• Stale policies are cleaned up periodically

CLI

• cf cli plugin help message should not have [] around port range

Chores

• Multiple policies about 1 app instance should only result in a single set-mark rule
• When the policy agent makes requests to the internal policy API it filters by apps that are on the cell

0.14.0

Netman is no more! The key change in this release is a rename from netman to cf-networking. This change is documented in the manifest changelog. At this point, there are no changes to manifest properties other than the release name.

We do not recommend using cf-networking-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com. Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

Significant Changes

Manifest Changes

• Rename netman to cf-networking for all container networking artifacts

Performance

• Compare c2c networking latency vs. router latency
• As PM I would like a measure of latency to apply a single additional policy with a number of existing policies
• As PM I would like to know how the effect of having a large ASG config on adding a single policy

Troubleshooting

• CF CLI Plugin respects CF_TRACE environment variable
• As a space developer I get a meaningful error message when I don't have network.write scope and try to configure a policy
• Add a Troubleshooting page for CF networking

Chores

• CF Networking Acceptance Tests using the same org causing failures

0.13.0

Key changes include support for self-service space developer configuration. A user can now request a network.write scope to configure policies for spaces where they have Space Developer privileges.
We do not recommend using netman-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com. Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

Significant Changes

Self-service for space developers

• Space developers with network.write scope can list policies for apps in spaces they own

Scalability and performance

• scalability tests should pass with 200 apps, 10 instances and 3 policies each
• IP tables for policies should not be re-written when policies have not changed

0.12.0

Key changes include configurable subnet ranges and masks, self service policy configuration and enhancements for reducing policy enforce time.

We do not recommend using netman-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com. Take a look at known issues for current limitations and known issues.

Verified with the following:

• CF deployment

Significant Changes

Deployment Changes

• The subnet and mask for the overlay network is configurable
• As an operator I would like to support more than 254 cells with legacy networking features

Space Developer self-service policy configuration

• Space developers with network.write scope can create policies using the API for apps in spaces they own
• Space developers with network.write scope can delete policies for apps in spaces they own

UX changes

• Remove old delete API endpoint

Performance and Scalability

• As an operator I don't expect iptables to be rewritten continuously when there are no policy changes

Miscellaneous Changes

• The eureka demo example uses the CF_INSTANCE_INTERNAL_IP environment variable
• Compare ingress networking performance with and without netman

0.11.0

Key changes include logging enhancements and UX changes to the DELETE and GET APIs.

We do not recommend using netman-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com.

Take a look at known issues for current limitations and known issues. Verified with the following:

• CF deployment

Significant Changes

Deployment Changes

• Use opsfile to add netman to cf-deployment
• Document deployment instructions with cf-deployment

Logging

• When a c2c connection between containers is allowed, I see a log line in syslog at the destination
• As an operator I can get all logs related to netman by using a keyword
• Reduce log message noise due to missing policy_group_id

UX changes

• As a CF operator I would like a delete policy API on GCP
• CF CLI calls the new delete API
• CLI plugin makes GET requests without a request body

0.10.0

Key changes include manifest changes related to policy server DB configuration, logging enhancements and testing related to data plane security.

We do not recommend using netman-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com.

Verified with the following:

• CF deployment

Significant Changes

Manifest Changes

• Policy server database can be configured through individual properties instead of an opaque string

Logging

• Log levels for vxlan-policy-agent are reconfigurable at runtime
• Logging for c2c iptables is reconfigurable at runtime
• Log levels for policy-server are reconfigurable at runtime

Security

• Move flannel state dir to something under /var/vcap
• As an attacker my containers can reach local addresses on the host VM
• Redact tokens/passwords in policy server log messages

Miscellaneous

• netman-release has a NOTICE file with license information
• Containers can be created while policy server is down and receive traffic when the policy-server comes back up
• Masquerade rule should be written by something other than vxlan-policy-agent
• SPIKE: Containers can connect to an IP address on the host

0.9.0

Key changes include using BulkNetOut for Garden external networker, logging and security enhancements and validation of policy server high availability.

We do not recommend using netman-release in production yet, but give it a try and give us your feedback in the #container-networking channel on cloudfoundry.slack.com.

Verified with the following:

• CF deployment

Follow the link above to get information about individual releases.

Significant Changes

Important fixes

• garden-external-networker down is idempotent

Scalability and Performance

• BulkNetOut should be fast while using the external networker
• Handle a bulkNetOut call from Garden to the external networker

Security

• Internal components should not rely on external load-balancer & DNS in order to boot - fix UAA
• Internal components should not rely on external load-balancer & DNS in order to boot - fix CC

High Availability

• Explore how to provide high availability for policy server

Logging

• vxlan policy agent log level should be configurable via bosh spec property
• Cats & Dogs Backend App should log requests
• policy server log level should be configurable via bosh spec property

Application Security Groups

• Enabling netman does not have a negative impact on ASGs configured in PWS