Ylem documentation
  • 🗒️General information
    • Introduction to Ylem
    • Quick start guide
    • Release notes
  • 🔬Open-source edition
    • Installation
    • Usage of Apache Kafka
    • Task processing architecture
    • Configuring integrations with .env variables
  • 💡Integrations
    • Connecting an integration
    • Library of integrations
      • Amazon Redshift
      • Apache Kafka
      • APIs
      • Atlassian Jira
      • AWS Lambda
      • AWS RDS
      • AWS S3
      • ClickHouse
      • ElasticSearch
      • E-mail
      • Google Big Query
      • Google Cloud SQL
      • Google Pub/Sub
      • Google Sheets
      • Immuta
      • Incident.io
      • Jenkins
      • Hubspot
      • Microsoft Azure SQL
      • MySQL
      • OpenAI ChatGPT
      • Opsgenie
      • PostgreSQL
      • PlanetScale
      • RabbitMQ
      • Salesforce
      • Slack
      • Snowflake
      • Tableau
      • Twilio. SMS
      • WhatsApp (through Twilio)
    • Initial demo data source
  • 🚡Pipelines
    • Pipeline management
    • Tasks
      • Aggregator
      • API Call
      • Code
      • Condition
      • External trigger
      • Filter
      • For each
      • GPT
      • Merge
      • Notification
      • Query
      • Pipeline runner
      • Processor
      • Transformer
    • Running and scheduling pipelines
    • Library of templates
    • Environment variables
    • Mathematical functions and operations
    • Formatting of messages
  • 📈Statistics and profiling
    • Statistics of runs
    • Slow tasks
  • 📊Metrics
    • Metric management
    • Using previous values of a metric
  • 💼Use cases, patterns, templates, examples
    • Use cases
    • Messaging patterns
      • Datatype Channel
      • Message Dispatcher
      • Messaging Bridge
      • Message Bus
      • Message Filter
      • Message Router
      • Point-to-Point Channel
      • Publish-Subscribe Channel
      • Pull-Push
    • Functional use cases
      • Streaming from Apache Kafka and messaging queues
      • Streaming from APIs
      • Streaming from databases
      • Data orchestration, transformation and processing
      • Usage of Python and Pandas
      • KPI Monitoring
      • OKRs and custom metrics
      • Data Issues & Incidents
      • Reporting
      • Other functional use cases
    • Industry-specific use cases
      • Finance and Payments
      • E-commerce & Logistics
      • Customer Success
      • Security, Risk, and Anti-Fraud
      • Anti-Money Laundering (AML)
  • 🔌API
    • OAuth clients
    • API Reference
  • 👁️‍🗨️Other resources
    • FAQ
    • Our blog on Medium
Powered by GitBook
On this page
  • User authentication with Google
  • OpenAI ChatGPT in pipelines
  • Using Twilio to send SMS from pipelines
  • Using AWS KMS to encrypt/decrypt data
  • Sending E-mails with Amazon SES
  • OAuth 2.0 Integrations
  • Atlassian Jira
  • Hubspot
  • Salesforce
  • Slack

Was this helpful?

Edit on GitHub
  1. Open-source edition

Configuring integrations with .env variables

PreviousTask processing architectureNextConnecting an integration

Last updated 6 months ago

Was this helpful?

The open-source edition of Ylem enables you to configure some integrations by creating your accounts there first and then using credentials such as tokens, API keys, or secret keys in Ylem's environment variables to make it work.

On this page, we describe all such integrations and the environment variables you need to configure to make them work. However, you can also find necessary information at the global .env.common file in the root folder or the .env files of every microservice.

User authentication with Google

If you want to enable user authentication in Ylem's UI through Google, you need to configure the following parameters in the .env file (./backend/users/.env if you build Ylem from the source):

USERS_GOOGLE_CLIENT_ID=
USERS_GOOGLE_CLIENT_SECRET=
USERS_GOOGLE_CALLBACK_URL=http://%%YOUR_DOMAIN_NAME_IS_HERE%%/auth/google/callback

Read the from Google to configure user authentication and obtain client ID and secret first.

OpenAI ChatGPT in pipelines

To be able to use task, you need to configure your API Secret Key and choose a model you want to use:

TASK_RUNNER_OPENAI_GPT_KEY=%%YOUR_KEY_IS_HERE%%
TASK_RUNNER_OPENAI_MODEL=gpt-4o-mini

The following two variables need to be placed in the.env file ( or ./processor/taskrunner/.envif you build Ylem from the source). Read more about and in the official OpenAI reference guide.

Using Twilio to send SMS from pipelines

If you want to , you need to create your own account and configure the following parameters:

YLEM_INTEGRATIONS_TWILIO_ACCOUNT_SID=
YLEM_INTEGRATIONS_TWILIO_AUTH_TOKEN=
YLEM_INTEGRATIONS_TWILIO_NUMBER_FROM=

The following variables need to be placed in the root folder .env file (or ./.env.commonif you build Ylem from the source).

Using AWS KMS to encrypt/decrypt data

Ylem supports integration with AWS KMS for encrypting/decrypting information about users, organizations, and integration credentials.

The following variables need to be placed in the root folder .env file ( or./.env.commonif you build Ylem from the source).

AWS_KMS_KEY_ID= 
AWS_REGION= 
AWS_ACCESS_KEY_ID= 
AWS_SECRET_ACCESS_KEY=

This integration is optional and if these variables are left empty, the information will be stored unencrypted.

Sending E-mails with Amazon SES

The same variables in the .env file (./env.common for building Ylem from the source) can be used to enable E-mail sending with Amazon SES:

AWS_REGION=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=

OAuth 2.0 Integrations

Atlassian Jira

The following parameters need to be changed on the .env file (orbackend/integrations/.env for building Ylem from the source):

INTEGRATIONS_JIRA_OAUTH_CLIENT_ID=%%REPLACE_IT_WITH_THE_CLIENT_ID_OF_YOUR_APP%%
INTEGRATIONS_JIRA_OAUTH_CLIENT_SECRET=%%REPLACE_IT_WITH_THE_CLIENT_SECRET_OF_YOUR_APP%%
INTEGRATIONS_JIRA_AFTER_AUTHORIZATION_REDIRECT_URL=https://%%REPLACE_IT_WITH_THE_DOMAIN_NAME_OF_YOUR_YLEM_INSTANCE%%/jira-authorizations/{uuid}/?justConnected

Hubspot

The following parameters need to be changed on the .env file (orbackend/integrations/.env for building Ylem from the source):

INTEGRATIONS_HUBSPOT_OAUTH_CLIENT_ID=%%REPLACE_IT_WITH_THE_CLIENT_ID_OF_YOUR_APP%%
INTEGRATIONS_HUBSPOT_OAUTH_CLIENT_SECRET=%%REPLACE_IT_WITH_THE_CLIENT_SECRET_OF_YOUR_APP%%
INTEGRATIONS_HUBSPOT_AFTER_AUTHORIZATION_REDIRECT_URL=https://%%REPLACE_IT_WITH_THE_DOMAIN_NAME_OF_YOUR_YLEM_INSTANCE%%/hubspot-authorizations/{uuid}/?justConnected

Salesforce

The following parameters need to be changed on the .env file (orbackend/integrations/.env for building Ylem from the source):

INTEGRATIONS_SALESFORCE_OAUTH_CLIENT_ID=%%REPLACE_IT_WITH_THE_CLIENT_ID_OF_YOUR_APP%%
INTEGRATIONS_SALESFORCE_OAUTH_CLIENT_SECRET=%%REPLACE_IT_WITH_THE_CLIENT_SECRET_OF_YOUR_APP%%
INTEGRATIONS_SALESFORCE_AFTER_AUTHORIZATION_REDIRECT_URL=https://%%REPLACE_IT_WITH_THE_DOMAIN_NAME_OF_YOUR_YLEM_INSTANCE%%/salesforce-authorizations/{uuid}/?justConnected

Slack

The following parameters need to be changed on the .env file (orbackend/integrations/.env for building Ylem from the source):

INTEGRATIONS_SLACK_OAUTH_CLIENT_ID=%%REPLACE_IT_WITH_THE_CLIENT_ID_OF_YOUR_APP%%
INTEGRATIONS_SLACK_OAUTH_CLIENT_SECRET=%%REPLACE_IT_WITH_THE_CLIENT_SECRET_OF_YOUR_APP%%
INTEGRATIONS_SLACK_AFTER_AUTHORIZATION_REDIRECT_URL=https://%%REPLACE_IT_WITH_THE_DOMAIN_NAME_OF_YOUR_YLEM_INSTANCE%%/salesforce-authorizations/{uuid}/?justConnected

Read more about setting up Twilio auth token and account side .

Accounts and API tokens can be viewed in the Twilio dashboard here:

More information about it can be found on the .

The full information on enabling Atlassian Jira integration for the open-source edition is available on the .

The full information on enabling Hubspot integration for the open-source edition is available on the .

The full information on enabling Salesforce integration for the open-source edition is available on the .

The full information on enabling Slack integration for the open-source edition is available on the .

🔬
following guide
GPT
API Keys
available models
send an SMS from the pipeline Notification task
Twilio
here
https://console.twilio.com/us1/account/keys-credentials/api-keys
integration page
integration page
E-mail integration page
integration page
integration page