Response_check= lambda response: True if 200 in response.status_code else False, dag=dag) The action operators reside under the module “airflow.operators”Īn example of Action Operator is SimpleHTTPOperator, let’s invoke a REST service and handle the response with DAG('getitems', start_date=datetime(2022, 1, 1), schedule_interval="*/5 * * * *") as dag: The naming convention in Airflow is very clean, simply by looking at the name of Operator we can identify under what category the Operator is. The Action Operators in Airflow are the Operators which are used to perform some action, like trigger HTTP request using SimpleHTTPOperator or execute a Python function using PythonOperator or trigger an email using the EmailOperator. Let’s explore each of them and what types of Operators falls under these categories Action Operator Operators in Airflow fall under three categories Action, Transfer, and Sensor. Upon looking closely, we can say that each step is a task in a DAG and all of them can be handled by specific operators in Airflow. Let’s understand by a use-case. Say the organization receives a CSV file every day on S3, then aggregation and normalization are of the CSV file is required say using Pandas and saves the data to the SQLite and after processing, email the engineering team about the completion. There are various in-built Operators in Airflow for performing specific tasks like PythonOperator which can be used to run a Python function, SimpleHTTPOperator can be used to invoke a REST API and handle responses, EmailOperator used to send an email and to interact with Databases there are several operators like MySQLOperator for MySQL, SqlliteOperator to interact with SQLite, PostgresOperator, OracleOperator.
In a DAG the tasks are executed through Operators, in Airflow multiple Operators together form a workflow and we can also define the execution order of various Operators.
The article explains briefly what Operators in Airflow are and the different types of Operators with example.
0 kommentar(er)
