如何消除气流中的缺省示例阻力

我是 Airbnb 开源工作流/数据管道软件 气流的新用户。在 WebUI 启动之后，有许多默认的示例日期。我尝试了很多方法来移除这些伤口，但都失败了。

  • load_examples = False设置为 airflow. cfg。
  • 删除文件夹 lib/python2.7/site-package/airflow/example _ dags。

在我删除 dags 文件夹之后，这些示例 dags 的状态被更改为灰色，但是这些项目仍然占据 web UI 屏幕。并且在 airflo.cfg 中将一个新的 dag 文件夹指定为 dags_folder = /mnt/dag/1。我检查了这个文件夹，什么都没有。我觉得很奇怪，为什么移除这些例子如此困难。

53442 次浏览
小开

When you startup airflow, make sure you set:

load_examples = False

inside your airflow.cfg

If you have already started airflow with this not set to false, you can set it to false and run airflow resetdb in the cli (!which will destroy all current dag information!).

Alternatively you can go into the airflow_db and manually delete those entries from the dag table.

小开

TL;DR: check that you have only DAG files in your dags_folder -- Airflow will traverse this directory recursively and try to load all .py files.

I've lost some time debugging similar behaviour of Airflow: even though load_examples = False, airflow was still loading tons of unnecessary stuff, including example_dags. The problem was that I was having a virtualenv directory venv/ in dags_folder, and -- I was not expecting that -- Airflow searches for dags recursively in the dags dir. So it was loading example_dags from the apache-airflow installed in that virtualenv.

UPD: there's a .airflowignore file to ignore directories from dags_folder

小开

If LDAP based authentication is turned on after airflow resetdb and restarting airflow your login window may not appear and may give an error due to cached login id ( but no password ). If this happens, clear your cache and try. If that still doesn't work, turn off authentication, stop airflow and start. Then turn on authentication stop and start - you will be able to see login window and login with your LDAP authentication -Suresh

小开

Like others have said, you can change load_examples = False within airflow.cfg. However this requires that the cfg file already existing.

You can init the airflow DB without having to configure the cfg file by using ENV variables.

export AIRFLOW__CORE__LOAD_EXAMPLES=False
airflow initdb

See docs for more information.

小开

Before you start airflow make sure you set load_example variable to False in airflow.cfg file. By default it is set to True.

load_examples = False

If you have already started airflow, you have to manually delete example DAG from the airflow UI. Click on delete icon available on the right side of the DAG to delete it.

Click on the delete icon available on the right side of the DAG to delete it

Instead of manually deleting example DAG, you can reset your database by using airflow resetdb command but that will delete your connections and variables and other important information. Do not use airflow resetdb option in production.

小开

For Airflow 2.0, in docker-compose.yaml you can set AIRFLOW__CORE__LOAD_EXAMPLES: 'false' to not load them instead of editing the .cfg file.

小开

Easy Way:

  1. put ..

    load_examples = False

into the airflow.cfg file

  1. then close and restart the webserver and scheduler
小开

while starting up the airflow make sure to change

load_examples = False

in the airflow.cfg file .Then close and restart the webserver and scheduler.


提交答案