chaostoolkit-kubernetes icon indicating copy to clipboard operation
chaostoolkit-kubernetes copied to clipboard

SSLCertVerificationError while trying to run experiment againts a remote k8s cluster

Open TwinkleTShah opened this issue 5 years ago • 1 comments

Steps:

  1. Installed chaostoolkit-kubernetes using pip install chaostoolkit-kubernetes command on my local windows machine
  2. Have a remote kubernetes cluster running. Added the kube config file pointing to the remote k8s cluster in C:\Users<xxx>.kube folder on local machine
  3. Verified kube config file by making running "kubectl" commands.
  4. Created a simple experiment as shown https://docs.chaostoolkit.org/drivers/kubernetes/#usage
  5. Execute chaos run by passing the experiment file.

Output: [2020-09-01 18:52:53 INFO] Validating the experiment's syntax [2020-09-01 18:52:53 INFO] Experiment looks valid [2020-09-01 18:52:53 INFO] Running experiment: Do we remain available in face of pod going down? [2020-09-01 18:52:53 INFO] Steady state hypothesis: Verifying service remains healthy [2020-09-01 18:52:53 INFO] Probe: all-our-microservices-should-be-healthy [2020-09-01 18:52:57 ERROR] => failed: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='xxxxxxxx', port=443): Max retries exceeded with url: /k8s/clusters/xxxxx/apis/apps/v1/namespaces/xxxxx/deployments?fieldSelector=metadata.name%3Dxxxxx (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1091)'))) [2020-09-01 18:52:57 WARNING] Probe terminated unexpectedly, so its tolerance could not be validated [2020-09-01 18:52:57 CRITICAL] Steady state probe 'all-our-microservices-should-be-healthy' is not in the given tolerance so failing this experiment [2020-09-01 18:52:57 INFO] Experiment ended with status: failed

Can someone help what could be missing?

TwinkleTShah avatar Sep 01 '20 13:09 TwinkleTShah

@dustinfarris I'm actually not using controllers at all (or rather not creating controller.js, just using what Ember auto-generates at runtime). Instead my route dispatches an action to fetch the initial list data in the model() hook, and all subsequent updates for paging, searching, sorting, etc occur within the component, which dispatches actions to the redux store. The route is then observing the store for changes and updating it's queryParams when appropriate. This makes for a better separation of concerns as the component could care less about query parameters and whether or not it is being rendered in it's own route. The only piece currently strongly tying the component to the route is the fact that the route makes the initial fetch.

import route from 'ember-redux/route'

import {updateListItems} from 'my-app/actions/items'

function getQueryParamsFromReduxStoreState (state) {
  return {
    limit: state.limit || 10,
    page: state.page || 0,
    query: state.query || '',
    sort: state.sort || 'name'
  }
}

export default route({
  queryParams: {
    limit: {refreshModel: false},
    page: {refreshModel: false},
    query: {refreshModel: false},
    sort: {refreshModel: false}
  },

  activate (dispatch, ...args) {
    this._super(...args)
    this.subscribeToReduxStore()
  },

  deactivate (dispatch, ...args) {
    this._super(...args)
    this.unsubScribeFromReduxStore()
  },

  model (dispatch, params, {queryParams}) {
    const {limit, page, query, sort} = queryParams

    dispatch(
      updateListItems({limit, page, query, sort})
    )
  },

  onReduxStoreChange (dispatch) {
    const state = this.get('redux').getState().listView
    const currentQueryParams = this.get('router.router.state.queryParams')
    const expectedQueryParams = getQueryParamsFromReduxStoreState(state)
    const expectedKeys = Object.keys(expectedQueryParams)

    if (expectedKeys.some((key) => currentQueryParams[key] !== expectedQueryParams[key])) {
      this.transitionTo(this.routeName, {
        queryParams: expectedQueryParams
      })
    }
  },

  subscribeToReduxStore () {
    this._unsubscribe = this.get('redux')
      .subscribe(this.onReduxStoreChange.bind(this))
  },

  unsubScribeFromReduxStore () {
    if (!this._subscription) return
    this._unsubscribe()
    this._unsubscribe = null
  }
})()

sandersky avatar Feb 19 '17 20:02 sandersky

I probably missed some prior discussion on this, but is this functionally equivalent to the syntax you'd use today? E.g.

import Ember from 'ember';
import route from 'ember-redux/route';

function afterModel(dispatch, model, transition) {
  if (Ember.isPresent(transition)) {
    dispatch({
      queryParams: Object.keys(this.get('queryParams')),
      type: 'SET_QUERY_PARAMS'
    });
  }
}

export default route({ afterModel })(Ember.Route.extend({
  queryParams: {
    foo: {
      refreshModel: false
    }
  }
}));

brettburley avatar Apr 05 '17 06:04 brettburley

I'm closing this out but I am reopening a PR w/ the tests + queryparams connect scenario (all code from this original PR).

I spent a good hour looking over the work here and decided not to pull this into the project. When I wrote this project the route helper was mostly a minimalist function that returned a new route w/ whatever model hook function(s) you passed in. I didn't want to hold state or do much of anything else for fear of having to maintain a fork in the road.

In situations like this (what Matt showed in the original PR) I would just build a normal ember route/ inject the redux service and use dispatch. This is a little more code but also it requires no work to keep it current. The future looks bright for service injection and this seems to be the happy path :)

That said, I did like the example Matt threw together and decided to PR 90% of his great work :)

toranb avatar Apr 28 '17 02:04 toranb

Are you using a self signed certificate? As a workaround, try to add the last line of the following snippet (insecure-skip-tls-verify) to your ~/.kube/config:

apiVersion: v1
kind: Config
clusters:
- name: "<my-cluster-name>"
  cluster:
    server: "https://<000.000.000.000>.xip.io/k8s/clusters/<x-x0x0x>"
    certificate-authority-data: "<blablabla>"
    insecure-skip-tls-verify: true

dargmuesli avatar Sep 09 '20 21:09 dargmuesli