How to add environment variables in Kubernetes config file?

I'm trying to use Kubernetes Go-Client in my program (which will run outside my Kubernetes cluster) which requires access to config file. The config file requires some token, service account details etc so that the program can access the Kubernetes cluster.

The config file is of the form below:

apiVersion: v1
kind: Config
users:
- name: testsa
  user:
    token: my-token
clusters:
- cluster:
    certificate-authority-data: my-cert
    server: my-server
  name: self-hosted-cluster
contexts:
- context:
    cluster: self-hosted-cluster
    user: testsa
  name: test-name
current-context: test-context

In the above file I need to give my-token, my-cert& my-server as environment variables since I can't hard-code this in the file present in the repository due to security reason. How can I do that?