Page MenuHomePhabricator

Improve the flink-app chart to provide more useful defaults
Closed, ResolvedPublic

Description

Writing a service relying on the flink-app chart requires setting many different values that sometimes have to copied in different places. It might make sense for the flink-app chart to be a more opinionated regarding the WMF usages of flink. This could be done by providing more useful defaults and/or standardized values placeholders to allow interpolating values managed by puppet more easily.

The config bits that could see improvements could be:

  • object storage configuration
  • h/a setup with zookeeper
  • kafka brokers

As of today writing a typical flink-app service requires the following setup:

app:
  image: docker-registry.discovery.wmnet/repos/my-team/my-job-image
  version: flink-1.16.2
  env:
    # disable JEMALLOC for testing
    - name: "DISABLE_JEMALLOC"
      value: "true"

  job:
    entryClass: org.wikimedia.flink.MyJob
    initialSavepointPath: s3://my-app-storage-bucket/savepoints/init_20230904
    jarURI: local:///opt/flink/my-job.jar
    upgradeMode: savepoint
    args: [ "/srv/app/conf/app-config.properties" ]
    state: running

  jobManager:
    replicas: 1
    resource:
      memory: 1600m

  taskManager:
    replicas: 3
    resource:
      memory: 3Gb
      cpu: 4

  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "4"
    high-availability: zookeeper
    high-availability.storageDir: s3://my-app-storage-bucket/zookeeper_ha_storage
    high-availability.zookeeper.quorum: flink-zk1001.eqiad.wmnet:2181,flink-zk1002.eqiad.wmnet:2181,flink-zk1003.eqiad.wmnet:2181
    high-availability.zookeeper.path.root: "/flink"
    state.savepoints.dir: s3://my-app-storage-bucket/savepoints
    state.checkpoints.dir: s3://my-app-storage-bucket/checkpoints
    s3.access-key: myapp:account
    s3.secret-key: secret
    s3.endpoint: thanos-swift.discovery.wmnet
    s3.path.style.access: "true"

  config_files:
    app-config.properties:
      config_flag1: foo
      config_flag2: bar
      brokers: kafka-jumbo1009.eqiad.wmnet:9092

kafka:
  allowed_clusters:
    - jumbo-eqiad

discovery:
  listeners:
    - mw-api-int-async-ro
    - schema
    - thanos-swift

mesh:
  enabled: true

But could be simplified to something like:

app:
  image: docker-registry.discovery.wmnet/repos/my-team/my-job-image
  version: flink-1.16.2
  env:
    # disable JEMALLOC for testing
    - name: "DISABLE_JEMALLOC"
      value: "true"

  job:
    entryClass: org.wikimedia.flink.MyJob
    jarURI: local:///opt/flink/my-job.jar
    initialSavepointPath: "/savepoints/init_20230904"
    upgradeMode: savepoint
    args: [ "/srv/app/conf/app-config.properties" ]
    state: running

  jobManager:
    replicas: 1
    resource:
      memory: 1600m

  taskManager:
    replicas: 3
    resource:
      memory: 3Gb
      cpu: 4

  flinkConfiguration:
    taskmanager.numberOfTaskSlots: "4"

  config_files:
    app-config.properties:
      config_flag1: foo
      config_flag2: bar
      brokers: "/srv/app/conf/jumbo-eqiad.properties" # "standardized" properties file holding bootstrap hosts

flink:
  object_store:
    swift_access_key: myapp:account
    swift_secret_key: secret
    swift_cluster: thanos-swift
    swift_bucket: my-app-storage-bucket
    savepoints_dir: "/savepoints"
    checkpoints_dir: "/checkpoints"
  high_availability:
    zookeeper_cluster: flink-eqiad
    zookeeper_path_root: "/flink"
    object_store_ha_path: "/zookeeper_path_root"

kafka:
  allowed_clusters:
    - jumbo-eqiad

discovery:
  listeners:
    - mw-api-int-async-ro
    - schema

mesh:
  enabled: true

The above is just an example to illustrate the idea mainly to reduce repeated values and might be flawed.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 959059 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/deployment-charts@master] Draft: Pull some flink config down into the chart

https://gerrit.wikimedia.org/r/959059

Change 960662 had a related patch set uploaded (by Bking; author: Ebernhardson):

[operations/puppet@production] k8s config: Provide kafka and zookeeper hostnames

https://gerrit.wikimedia.org/r/960662

Change 959059 merged by jenkins-bot:

[operations/deployment-charts@master] Pull some flink config down into the chart

https://gerrit.wikimedia.org/r/959059

Change 963731 had a related patch set uploaded (by Bking; author: Bking):

[operations/deployment-charts@master] flink-app: redeploy from new savepoint

https://gerrit.wikimedia.org/r/963731

Change 963731 merged by jenkins-bot:

[operations/deployment-charts@master] flink-app: redeploy from new savepoint

https://gerrit.wikimedia.org/r/963731