well after remove all deps in the requirements , the worker in mwaa run normally , now can try test the bad deps
I have a problem parsing DAG with error:
Broken DAG: [/usr/local/airflow/dags/test.py] No module named 'airflow.providers'
I added apache-airflow-providers-databricks to requirements.txt, and see from the log that:
Successfully installed apache-airflow-2.0.1 apache-airflow-providers-databricks-1.0.1 apache-airflow-providers-ftp-1.0.1 apache-airflow-providers-http-1.1.1 apache-airflow-providers-imap-1.0.1 apache-airflow-providers-sqlite-1.0.2 apispec-3.3.2 attrs-20.3.0 cattrs-1.3.0 clickclick-20.10.2 commonmark-0.9.1 connexion-2.7.0 flask-appbuilder-3.1.1 flask-caching-1.10.0 gunicorn-19.10.0 importlib-resources-1.5.0 inflection-0.5.1 isodate-0.6.0 marshmallow-3.10.0 marshmallow-oneofschema-2.1.0 openapi-schema-validator-0.1.4 openapi-spec-validator-0.3.0 pendulum-2.1.2 python-daemon-2.3.0 rich-9.2.0 sqlalchemy-jsonfield-1.0.0 swagger-ui-bundle-0.0.8 tenacity-6.2.0 termcolor-1.1.0 werkzeug-1.0.1
But the scheduler seems to be stuck:
The scheduler does not appear to be running. Last heartbeat was received 19 hours ago.
How can I restart it?
AWS Managed Airflow - how to restart scheduler?