Scheduler AppΒΆ
Scheduler App service to manage and execute jobs
- Schedule jobs, interval or crontab
- Requests chain jobs
- Modules
- Webhook: Call URL request
- Connections: Call Crawler task
Scheduler use apscheduler to control scheduler jobs, Apscheduler documentation
Installation with python 3
- Python >3.4
- RabbitMQ
- MongoDB
Download de repository
git clone https://github.com/maestro-server/scheduler-app.git
Important topics
Celery Beat consult schedulers collection in mongodb every 5 seconds and updated time to call the tasks.
Have 2 tasks called by beat
- webhook: Call HTTP request accordly arguments.
- connection: Consulting connection data, after call webhook.
Have support tasks called by outhers tasks.
- chain and chain_exec: Called by webhook, this create another job after the first finish.
- depleted_job: If any job recevied something wrong, this taks is called e depleted that job.
- notify_event: Send notification event.
Installation with python 3
- Python >3.4
- RabbitMQ
- MongoDB
Download de repository
git clone https://github.com/maestro-server/scheduler-app.git
Install run celery beat
celery -A app.celery beat -S app.schedulers.MongoScheduler --loglevel=info
or
npm run beat
Install run rabbit workers
celery -A app.celery worker -E --hostname=scheduler@%h --loglevel=info
or
npm run celery
Env variables
Env Variables | Example | Description |
---|---|---|
MAESTRO_DATA_URI | http://localhost:5010 | Data Layer API URL |
MAESTRO_DISCOVERY_URI | http://localhost:5000 | Discovery App URL |
MAESTRO_ANALYTICS_URI | http://localhost:5020 | Analytics App URL |
MAESTRO_REPORT_URI | http://localhost:5005 | Reports App URL |
MAESTRO_MONGO_URI | localhost | MongoDB URI |
MAESTRO_MONGO_DATABASE | maestro-client | Mongo Database name |
CELERY_BROKER_URL | amqp://rabbitmq:5672 | RabbitMQ connection |
MAESTRO_SECRETJWT_PRIVATE | XXX | Secret Key - JWT private connections |
MAESTRO_NOAUTH | XXX | Secret Pass to validate private connections |