Skip to content

Qarik-Group/service-fabric-boshrelease

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

BOSH release for service-fabric

This BOSH release and deployment manifest deploy a linux based standalone cluster of service-fabric.

Deployment

This repository includes base manifests and operator files. They can be used for initial deployments and subsequently used for updating your deployments.

Setting up a BOSH director

For those not familiar with BOSH or who do not have a running BOSH director, check out the guides on bosh.io for setting up the CLI and directors on AWS, Azure, GCP, vSphere, and more.

Stemcells

The ServiceFabric bosh-release requires the use of the newer Xenial (Ubuntu 18.04) stemcells. Other stemcells are currently not supported but may be considered in future releases. TO upload a stemcell, target and authenticate with your BOSH environment and upload a Xenial stemcell which are available here. Ensure you grab the latest Full Stemcell for your IaaS of choice.

Examples for vSphere, Azure, and AWS are:

export BOSH_ENVIRONMENT=<bosh-alias>
bosh upload-stemcell https://bosh.io/d/stemcells/bosh-vsphere-esxi-ubuntu-xenial-go_agent?v=97.16
bosh upload-stemcell https://bosh.io/d/stemcells/bosh-azure-hyperv-ubuntu-xenial-go_agent?v=97.16
bosh upload-stemcell https://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-xenial-go_agent?v=97.16

Cloud-Config

The base manifest assumes you have a service-fabric network defined as well as a service-fabric vm_type in your cloud-config as shown below with snippets from AWS and vSphere cloud-configs.

AWS specfic:

networks:
- name: service-fabric
  subnets:
  - azs:
    - z1
    - z2
    - z3
    cloud_properties:
      subnet: subnet-032d1aeeac2006243
    dns:
    - 8.8.8.8
    - 8.8.4.4
    gateway: 172.16.1.1
    range: 172.16.0.0/17
    reserved:
    - 172.16.0.0-172.16.0.10
vm_types:
- cloud_properties:
    ephemeral_disk:
      size: 20480
    instance_type: m4.xlarge
    root_disk:
      size: 51200
  name: service-fabric

vSphere Specific:

- name: service-fabric
  subnets:
  - azs:
    - z1
    - z2
    - z3
    cloud_properties:
      name: VMNetwork
    dns:
    - 8.8.8.8
    - 8.8.4.4
    gateway: 172.16.1.1
    range: 172.16.0.0/17
    reserved:
    - 172.16.0.0-172.16.0.10
  type: manual

vm_types:
- cloud_properties:
    cpu: 2
    disk: 51200
    ram: 8192
  name: service-fabric

Note: The above examples are just snippets, not complete cloud configs.

While the above snippets are mostly the same, the IaaS dependent properties listed under cloud_properties are what differentiate most cloud-configs from one another. For more information on the specific cloud_properties youll need to reference for your IaaS of choice, visit the cloud-config documentation here under the Cloud Providers section for AWS, Azure, GCP, vSphere, and more. For each IaaS be sure to look at the Example Cloud Config to base yours off of and ensure you have all of the required sections.

Deploying ServiceFabric

To deploy ServiceFabric, the easiest menthod would be to use the base manifest provided in this repo and customize it to fit your needs. The base manifest will deploy a 5 node cluster without authentication.

An example of a manifest including all of the valid properties is shown below with descriptions of all of the parameters and their default values (if applicable) below that.

---
name: service-fabric

instance_groups:
- name: service-fabric
  azs: [z1,z2,z3]
  instances: 5
  vm_type: service-fabric
  stemcell: default
  networks: [{name: service-fabric}]
  env:
    bosh:
      ipv6: {enable: true}
      swap_size: 0
  jobs:
  - name: service-fabric
    release: service-fabric
    properties:
      PrimaryAccountNTLMPasswordSecret: Secret-1
      SecondaryAccountNTLMPasswordSecret: Secret-2
      ClientConnectionEndpoint: 19000
      LeaseDriverEndpoint: 19001
      ClusterConnectionEndpoint: 19002
      ServiceConnectionEndpoint: 19006
      HttpGatewayEndpoint: 19080
      HttpApplicationGatewayEndpoint: 19081
      ApplicationEndpoints:
        Start: 22001
        End: 23000
      CpuPercentageNodeCapacity: 0.8
      MemoryPercentageNodeCapacity: 0.8
      MinLoadBalancingInterval: 5
      TargetReplicaSetSize: 4
      MinReplicaSetSize: 3
      SeedNodes: 5
      TLS:
        enable: true
        cluster_certificates:
          primary:
            certificate: |
              -----BEGIN CERTIFICATE-----
              -----END CERTIFICATE-----
            key: |
              -----BEGIN RSA PRIVATE KEY-----
              -----END RSA PRIVATE KEY-----
          secondary:
            certificate: |
              -----BEGIN CERTIFICATE-----
              -----END CERTIFICATE-----
            key: |
              -----BEGIN RSA PRIVATE KEY-----
              -----END RSA PRIVATE KEY-----
        admin_client_certificates:
          name-of-admin-user:
            certificate: |
              -----BEGIN CERTIFICATE-----
              -----END CERTIFICATE-----
        client_certificates:
          name-of-read-only-user:
            certificate: |
              -----BEGIN CERTIFICATE-----
              -----END CERTIFICATE-----
  - name: repair
    release: service-fabric
    properties: {}

- name: smoke-test
  azs: [z1]
  instances: 1
  lifecycle: errand
  vm_type: small
  stemcell: default
  networks: [{name: service-fabric}]
  env:
    bosh:
      ipv6: {enable: true}
  jobs:
  - name: smoke-test
    release: service-fabric

- name: upgrade
  azs: [z1]
  instances: 1
  lifecycle: errand
  vm_type: small
  stemcell: default
  networks: [{name: service-fabric}]
  env:
    bosh:
      ipv6: {enable: true}
  jobs:
  - name: upgrade
    release: service-fabric
    properties:
      FabricCodeVersion: 6.3.129.1
Property Description Default
PrimaryAccountNTLMPasswordSecret The password secret which used as seed to generate a password when using NTLM authentication. Generated in Manifest
SecondaryAccountNTLMPasswordSecret The password secret which used as seed to generate a password when using NTLM authentication. Generated in Manifest
ClientConnectionEndpoint Port to listen on for Client Connections 19000
LeaseDriverEndpoint Port to listen on for Lease Driver Connections 19001
ClusterConnectionEndpoint Port to listen on for Cluster Connections 19002
ServiceConnectionEndpoint Port to listen on for Service Connections 19006
HttpGatewayEndpoint Port to listen on for HTTP API connections 19080
HttpApplicationGatewayEndpoint Port for the HTTP Application Gateway (reverse proxy) to listen on - not currently supported on linux clusters but included for future implementation 19081
ApplicationEndpoints.Start Starting port for applications 22001
ApplicationEndpoints.End Ending port for applications 23000
TLS.enable Enable x509 certificate authentication for service-fabric false
TLS.cluster_certificates TLS private key (PEM encoded), used for intra Cluster communication, HTTPS API, and Web UI. Each element in the array is an object containing the field 'certificate', each of which supports a PEM block. N/A
TLS.admin_client_certificates Array of Certificates used by clients in admin role. Each element in the array is an object containing the field 'certificate', each of which supports a PEM block. N/A
TLS.client_certificates Array of Certificates used by clients in read-only role. Each element in the array is an object containing the field 'certificate', each of which supports a PEM block. N/A
CpuPercentageNodeCapacity Percentage of Node CPU Service Fabric processes are allowed to use. 0.8
MemoryPercentageNodeCapacity Percentage of Node Memory Service Fabric processes are allowed to use. 0.8
MinLoadBalancingInterval Defines the minimum amount of time that must pass before two consecutive balancing rounds. 5
TargetReplicaSetSize The number of replica sets for each partition. Increasing the number of replica sets increases the level of reliability for information; decreasing the chance that the information will be lost as a result of node failures; at a cost of increased load on Service Fabric and the amount of time it takes to perform updates. #ofNodes-1 (up to 7 nodes)
MinReplicaSetSize Defines the minimum number of replicas required to write into to complete an update. This value should never be more than the TargetReplicaSetSize. 3 or 5 depending on Cluster Size
SeedNodes Defines the number of seed nodes to be created with the cluster. By default this is 5 and the recommended value is up to 11. Note: If you plan on scaling down ensure that half+1 of the seed nodes remain in order to maintain quorum and avoid cluster collapse. 5

Finally after filling out the manifest according to your needs, bosh deploy the manifest via the following:

export BOSH_ENVIRONMENT=<bosh-alias>
export BOSH_DEPLOYMENT=service-fabric
git clone https://github.com/cloudfoundry-community/service-fabric-boshrelease.git
bosh deploy service-fabric-boshrelease/manifests/service-fabric.yml

Smoke Tests

An errand called smoke-test is provided to test core functionality once the cluster is deployed. The tests include deploying a Java, C# .NET Core, and Docker conatiner app and ensuring that they come up in a healthy state. NOTE:The container smoke test will fail if inside an air-gapped network or one which requires a proxy to access the internet. This is due to attempting to pull an image from dockerhub.

To run the smoke-test errand execute the following commmands:

export BOSH_ENVIRONMENT=<bosh-alias>
export BOSH_DEPLOYMENT=service-fabric
bosh run-errand smoke-test

Service Fabric Upgrades

To upgrade ServiceFabric to a new version, there is also an included errand called upgrade that by default will try to update the cluster to the newest available FabricCodeVersion. Depending on cluster size, this process can take a lot of time and can be monitored from the ServiceFabric WebUI.

To upgrade ServiceFabric to a specific version instead of the latest, you must update the upgrade errand job in the manifest and redeploy before running the errand.

Example configration for a specific version is shown below (Specifically the FabricCodeVersion property):

- name: upgrade
  azs: [z1]
  instances: 1
  lifecycle: errand
  vm_type: small
  stemcell: default
  networks: [{name: service-fabric}]
  env:
    bosh:
      ipv6: {enable: true}
  jobs:
  - name: upgrade
    release: service-fabric
    properties:
      FabricCodeVersion: 6.3.129.1

To run the upgrade errand execute the following commmands:

export BOSH_ENVIRONMENT=<bosh-alias>
export BOSH_DEPLOYMENT=service-fabric
bosh run-errand upgrade

Bosh Upgrades

When new versions of service-fabric-boshrelease are released the manifests/service-fabric.yml file will be updated. This means you can easily git pull and bosh deploy to upgrade to a new version of the bosh-release. New versions of the release will include updates for new stemcell versions as well as the most recent version of ServiceFabric.

export BOSH_ENVIRONMENT=<bosh-alias>
export BOSH_DEPLOYMENT=service-fabric
cd service-fabric-boshrelease
git pull
bosh deploy manifests/service-fabric.yml