Handle Regular Tasks with Symfony's Scheduler Component
Time to read:
Handle Regular Tasks with Symfony's Scheduler Component
When it comes to automating recurring tasks, one can be forgiven for thinking Cron jobs are as good as it gets. I recently had to automate a recurring task with a frequency of “it depends”, and while not impossible it was definitely challenging. So imagine my delight when I discovered the Symfony Scheduler component, which takes Cron jobs to a whole new level.
In this tutorial, I will show you how to use the Scheduler component to handle recurring tasks which don’t have straightforward frequencies.
What you will build
To demonstrate this, you will build a factory simulator. In terms of operations, the factory runs six days a week (Monday through Saturday). And, in addition to Sundays, the factory doesn’t run on Christmas Eve, Christmas Day, New Year’s Eve, and New Year's Day. On the days when the factory runs, it operates on 4-hour shifts starting at midnight with a 2-hour break between each shift.
To keep track of things, the factory requires reports to be generated and sent to management via email. In particular, three reports are required:
- A production report which is expected at the end of each day
- An incident report which is expected after every shift
- A compensation report which is expected on the last day of the month
Generating these reports is already tricky as it is, but with the added complexity of break times and down times (when the factory is closed) you can see why the process will be uncomfortable to handle with standard Cron jobs.
Prerequisites
To follow this tutorial, you will require the following.
- A basic understanding of and familiarity with PHP and Symfony
- Understanding of Cron jobs
- PHP 8.4 or above
- Composer globally installed
- The Symfony CLI
- A SendGrid account with a verified sender identity. If you are new to SendGrid, click here to create a free account
Create a new project
Create a new Symfony project in a directory named scheduler_demo and change into the new project directory, using the following commands:
Next, add the project’s dependencies using the following commands:
Docker won’t be used in this tutorial, so press the "x" key when prompted to create a compose .yaml file.
Here’s what each dependency is for:
- Cron-expression: This is required by the Symfony scheduler to parse Cron expressions
- Doctrine: This package will be used to handle database-related activity. The fixtures bundle is added as a dev dependency to simplify the process of seeding the database
- Faker: This is used to generate random data. For demo purposes, it is added as a project dependency (more often than not, this is better as a dev dependency)
- Mailer: This bundle will be used to send emails. It will be used in conjunction with the SendGrid Mailer Bridge component and the Symfony HTTP Client component
- Maker: This bundle helps with the auto-generation of code associated with entities, messages, and schedules
- Messenger: This adds Symfony’s messenger component to the project. This is used in your code (and by the scheduler) to dispatch messages to the message queue. For this tutorial, Doctrine will serve as the message transport hence the symfony/doctrine-messenger dependency
- PHPSpreadsheetBundle: This bundle integrates your application with the PhpSpreadsheet productivity library, allowing you to create spreadsheets
- Scheduler: This adds the scheduler component to your project. This component depends on Symfony’s Serializer which is also installed
Set up SendGrid API Key
As mentioned earlier, SendGrid will be used to handle emails. You will do this by making a request to the SendGrid API. However, the SendGrid API requires an API Key in the request header before it can be accepted. In this section, you will create a new key which can be used in the Symfony application. If you already have one, you can skip to the next section.
In your SendGrid console, click on the Create API Key button to create a new key. For security purposes, the key to be created will have restricted access. This limits the impact on your SendGrid account in the event that your key is leaked.
To do this, select the Restricted access option under API Key Permissions. In the Access Details section, select Full Access for Mail Send as shown below.
Click the Create & View button in order to view your API Key.The next screen will show your API key.
Set the required environment variables
In the project's top-level folder, create a new file named .env.local.
Then, update the relevant values in .env.local as shown below.
For ease of use, this tutorial uses SQLite as the database engine. However you can use any engine of your choice by using an appropriate format in the DATABASE_URLvariable.
In the MAILER_DSN variable, replace the YOUR_SENDGRID_API_KEY placeholder with the SendGrid API key you generated earlier. Next, replace the SENDER_EMAIL placeholder value with an email that matches a verified Sender Identity in your account . This is the email address that will show up in the “from ” section of the email. Next, replace the RECIPIENT_EMAIL placeholder with the email address you want the emails sent to.
Next, set up the database using the following command.
Set up the Messenger transports
By default, messages are handled as soon as they are dispatched. Depending on the volume of messages generated, this could pose a bottleneck to the system. An easy way of handling this is to queue the messages to a transport and handle them in the background via a worker.
However, with this system you may encounter issues tracking messages that weren’t handled successfully. This is because once a message has been handled, it is removed from the queue whether it was successful or not. You can manage this by also creating a transport where failed messages can be sent to for subsequent investigation and resolution.
Set up an asynchronous transport and a failure transport by updating config/packages/messenger.yaml to match the configuration shown below.
Set up your transports using the following command.
Finish the project setup
Finally, create the folders where reports will be saved to. In the project's top-level folder, create a new folder named reports, and in it three new folders named compensation, incident, and production.
Create a mailing service
Having set up the project, it’s time to build the features of the application - starting with a mailing service. In the src folder, create a new folder named Helper which will contain helper services the application will use - such as the mailing service. In the Helper folder, create a new folder named Mailing and in it a file named MailService.php with the following code in it.
The MailService has three constructor arguments:
- The Symfony Mailer instance, which sends the mail
- The sender's email address, which is retrieved from your environment variables via the Autowire attribute
- The recipient's email address, which is retrieved from your environment variables via the Autowire attribute
This service has one function named sendMail() which takes the email subject and message content. This service also includes attachments in the email. To do this it requires the path to the attachment, and the name of the file. Using these, it creates a new Email object, which is sent via the mailer.
Create the entities
In the src/Entity folder, create a new file named ProductType.php and add the following code to it.
Next, in the same directory, create another file named IncidentType.php and add the following code to it.
Next, create an entity to represent a worker using the following command.
Press Enter to skip adding fields via the command line.
Open the newly created src/Entity/Worker.php file and update its code to match the following.
Next, create an entity to represent a product using the following command.
Press Enter to skip adding fields via the command line. Then, open the newly created src/Entity/Product.php file and update its code to match the following.
You also need to add a function to ProductRepository, which allows you to retrieve products for a given date. Open src/Repository/ProductRepository.php and update its code to match the following.
Next, create an entity to represent an incident using the following command.
Press Enter to skip adding fields via the command line.
Open the newly created src/Entity/Incident.php file and update its code to match the following.
You also need some repository functions for the incident entity. Open src/Repository/IncidentRepository.php and update its code to match the following.
The getUncompensatedIncidents() function is used to retrieve the list of incidents for which the victim is yet to be compensated while the getIncidentsBetween() function takes two DateTime objects and returns all the incidents that occurred within that period.
Next, create an entity to represent the compensation to a worker for an incident using the following command.
Press Enter to skip adding fields via the command line. Then, open the newly created src/Entity/Compensation.php file and update its code to match the following.
Update the app fixture
Open src/DataFixtures/AppFixtures.php and update the code to match the following.
Using this fixture, you will be able to seed your database with 100 new workers.
Set up the database
Having created your entities and done your initial configuration, it’s time to create your database and seed it with initial data. To do this, run the following commands:
Create the helpers
The next thing you will do is build some helper services, which will be used with generating reports as well as determining whether the factory is operational.In the src/Helper folder, create a new file named FactoryOperationsHelper.php and add the following code to it.
This service contains two static functions which are used to determine whether the factory is operational. isBreakTime() takes a DateTime and checks if the factory is on break at that time, while isDownTime() takes a DateTime instance and determines whether the factory is operational that day.
Create the report writers
Next, in the src/Helper folder, create a new folder named Reporting. Then, in the src/Helper/Reporting folder, create a new file named AbstractReportWriter.php and add the following code to it.
This class serves as the base implementation for a report writer and holds all the functionality related to writing a report. It also contains an abstract function named write() which must be implemented by child classes. The constructor takes the location where the file should be saved. For each report you will write, you will extend the AbstractReportWriter and add the report-specific code in it.
Next, create a new file named CompensationReportWriter.php in the src/Helper/Reporting folder and add the following code to it.
As explained earlier, this service extends the AbstractReportWriter class you created earlier. Using the Autowire attribute, the save location is specified in the service constructor. Using the functionality provided in the parent class, the appropriate cells are populated and formatted, and the resulting spreadsheet is saved using the month and year of creation as the file name.
Next, create a new file named IncidentReportWriter.php in the src/Helper/Reporting folder and add the following code to it.
The process flow is the same as that of the CompensationReportWriter — autowire the save location in the constructor, overwrite the write() function, and populate the appropriate cells using the parent functions. Because this report is generated multiple times in the same day, the day and hour of generation is used as a filename.
The last report writer you need is for the production report. Create a new file named ProductionReportWriter.php in the src/Helper/Reporting folder and add the following code to it.
Create messages
The next step is to create the messages to simulate production, which will be dispatched repeatedly by the scheduler. To do that, use the following command.
When prompted with the message below, select "1" for the async transport.
Next, open the newly created src/MessageHandler/GenerateProductsHandler.php file and update its code to match the following.
The GenerateProducts message is dispatched to simulate the production of items on the factory line. Whenever a GenerateProducts message is dispatched, the message handler creates a new product entry for each of the product types with a random quantity. This entry is persisted to the database and all the changes are saved.
Next, create the message to simulate the occurrence of incidents using the following command.
When prompted with the message below, select "1" for the async transport.
Next, open the newly created src/MessageHandler/GenerateIncidentsHandler.php file and update its code to match the following.
When a GenerateIncidents message is dispatched, a random number of workers are retrieved and random incidents are created. This simulates the occurrence of incidents on the factory line. These incidents are then saved to the database.
Next, create the message to generate the production report using the following command.
When prompted with the message below, select "1" for the async transport.
Next, open the newly created src/MessageHandler/GenerateProductionReportHandler.php file and update its code to match the following.
When the GenerateProductionReportHandlerreport is dispatched, the products created the previous day are retrieved from the database using the getProductionForDate() function you declared earlier in the ProductRepository and passed to the ProductionReportWriter for report generation. Once the report has been generated, an email is sent with the report included as an attachment.
Next, create the message to generate the incident report using the following command.
When prompted with the message below, select "1" for the async transport.
Next, open the newly created src/MessageHandler/GenerateIncidentReportHandler.php file and update its code to match the following.
Similar to the GenerateProductionReportmessage, when this message is dispatched, the system retrieves incidents from the last four hours and prepares a report using the respective repository and report writer service. The generated report is immediately sent as an email attachment to management.
Next, create the message to generate the compensation report using the following command.
When prompted with the message below, select "1" for the async transport.
Next, open the newly created src/MessageHandler/GenerateCompensationReportHandler.php file and update its code to match the following.
When this message is dispatched, the uncompensated incidents are retrieved from the database. For each of the affected workers, a Compensation object is either retrieved or created, and the incidents are compiled to determine the final compensation due to the employee. After saving the updated compensations, they are passed on for report writing and the generated report is subsequently sent to management.
Create the triggers
Every recurring message requires a trigger which determines its frequency. In addition to that, you can decorate triggers to either introduce randomness, or skip execution at a time (as applies to this use case). It doesn’t stop there, however, as you can create your custom triggers which allow you specify the next execution time.
Start by creating the custom triggers. In the src folder, create a new folder named Trigger. Next, create a new file named ExcludeBreaksTrigger.php and add the following code to it.
Because this trigger is a decorator for another trigger, the constructor function declares the inner trigger as a parameter. Every custom trigger must implement the TriggerInterface. This requires an implementation of the __toString() function which returns a readable representation of the trigger, and the getNextRunDate() function which is used to determine the next run date.
In this case, the next run date cannot be during a break, so a while loop is used to update the next run date until a date outside the break period is reached. It updates the next run date by calling the getNextRunDate() function of the inner trigger.
Next, create another file named ExcludeDownTimeTrigger.php in the src/Trigger folder and add the following code to it.
This trigger is similar to the ExcludeBreaksTriggerexcept that, in this case, it ensures that the next run date does not fall within a period when the factory is shut down.
Update the task schedule
With the custom triggers in place, it’s time to update the schedule that orchestrates all the tasks. Open src/Schedule.php and update its code to match the following.
The Schedule class implements the ScheduleProviderInterface, which requires implementing the getSchedule() function. This function returns a Schedule object containing the messages to be dispatched recurrently. The returned schedule is stateful, so as to avoid disruptions in the event that the message consumer is restarted.
The returned schedule contains five recurring messages:
- The recurring message to generate the compensation report. This message will always run on the last day of the current month. Observe how you don’t need to add any extra logic for whether the month in question is February (in a leap year or not), or the number of days the month in question has.
- The recurring message for generating the incident report. This message is generated from a trigger. This trigger is a
CronExpressionTriggerprovided by Symfony which allows you to create a trigger from a Cron entry. This will run every 6 hours (except for days when the factory is closed). - The recurrent message for generating the products. This message is also generated from a
CronExpressionTrigger. However, instead of a Cron entry, the specification for this trigger is@midnight. This is an alias provided by Symfony to create the Cron entry easily. This will run at midnight every day (except on days when the factory is closed). - The recurrent message responsible for generating products. This is also generated from a
CronExpressionTriggeralbeit with a different alias. This alias means that the message will be dispatched at a random minute every hour. - The recurrent message responsible for generating incidents. This is similar to the product generation trigger
Test your implementation using the following command.
The response should match the image shown below:
To run the scheduler transport, run the following command
It is important to note that the scheduler transport only dispatches messages which should be handled either synchronously or by another transport (in this case the async transport). To handle the messages dispatched by the scheduler transport, run the following command in another terminal.
That's how to handle regular tasks with Symfony's Scheduler component
There you have it! In this tutorial I showed you how the Scheduler component makes managing nuanced recurring tasks easier but that’s not the only benefit. With this, you don’t have to worry about managing crontab entries, a potential pain point when managing containerized applications and clusters which scale dynamically.
If you’re wondering how to deploy this, have a look at the Messenger documentation. Just remember that you will need to set up two workers - one for the async transport and a second for the scheduler transport.
You can review the final codebase for this article on GitHub, should you get stuck at any point. I’m excited to see what else you come up with. Until next time, make peace not war✌🏾
Joseph Udonsak is a software engineer with a passion for solving challenges — be it building applications, or conquering new frontiers on Candy Crush. When he’s not staring at his screens, he enjoys a cold beer and laughs with his family and friends. Find him at LinkedIn , Medium , and Dev.to .
Related Posts
Related Resources
Twilio Docs
From APIs to SDKs to sample apps
API reference documentation, SDKs, helper libraries, quickstarts, and tutorials for your language and platform.
Resource Center
The latest ebooks, industry reports, and webinars
Learn from customer engagement experts to improve your own communication.
Ahoy
Twilio's developer community hub
Best practices, code samples, and inspiration to build communications and digital engagement experiences.