ASP.NET

Hangfire and Setup

What Background Processing?, And Why?

A background process is a program that is running without user input. A number of background processes can be running on a multitasking operating system, while the user is interacting with the foreground process Some background processes, such as data processing, Invoice generation or email notification for example, never require user input. Others are merely in the background temporarily while the user is busy with the program presently running in the foreground. So that other process can be sleeping and taking up swap space, until activated, which thus makes it currently a background process.

So in simple term “the ability of a system to perform a low-priority task while, at the same time, dealing with a main application”.

Ways to implement background processing?

In order to perform background processing we have multiple ways to accomplish, below are few of the .Net in-build features:

  1. TPL (Task Parallel Library) :

The TPL allows us to deal with tasks as code blocks and handing the tasks off to an engine that manages threads for us. The abstraction of the TPL positions our code to scale up to machines with multiple cores. Writing code to the TPL allows us to allow the TPL engine to worry about multi-core issues for us. The TPL allows us to scale LINQ queries to a multicore level without drastically changing the syntax of the LINQ query itself. Other than adding the .AsParallel() extension method call, PLINQ queries look like LINQ queries.

 

But, it doesn’t free you from forgetting completely about multithread issues. Shared state still has to be locked, or otherwise managed, and thread-safe data structures have to be used in favor of thread-unsafe data structures. Thread safety becomes a paramount concern. Even though you don’t have to manage the threads yourself, you won’t be freed from having to consider thread safety. This will hang the entire app and also user’s device, which not good practice.

 

  1. QueueBackgroundWorkItem :

QueueBackgroundWorkItem schedules a task which can run in the background, independent of any request. This differs from a normal ThreadPool work item in that ASP.NET automatically keeps track of how many work items registered through this API are currently running, and the ASP.NET runtime will try to delay AppDomain shutdown until these work items have finished executing. This was specifically added to enable ASP.NET apps to reliably run short-lived background tasks with some limitations.

 

The QueueBackgroundWorkItem API cannot be called outside of an ASP.NET-managed AppDomain.

The AppDomain shutdown can only be delayed 90 seconds. If you have so many items queued that they can’t be completed in 90 seconds, the ASP.NET runtime will unload the AppDomain without waiting for the work items to finish.

The code that we run in the background thread doesn’t have access to commonly used context properties. If you need HttpContext information, copy the values we care about to a state object or inside a closure and pass it in to the background worker.  Don’t pass the HttpContext instance itself, as it’s not a thread-safe object and even simple property getters  might throw.

Scheduled work items are not guaranteed to ever execute, once the app pool starts to shut down, QueueBackgroundWorkItem calls will not be honored.

We don’t guarantee that background work items will ever get invoked or will run to completion.  For instance, if we believe a background work item is misbehaving, we’ll kill it.  And if the w3wp.exe process crashes, all background work items are obviously dead.  If you need reliability, you should use Azure’s built-in scheduling functions.

 

Code Snippet: (The API is pretty straightforward, taking  Func<CancellationToken, Task>. Here’s an example that kicks of a background work item from an MVC action)

 

 

  1. Quartz Enterprise Scheduler .NET :

Quartz.NET is a .NET port of the popular Java job scheduling framework of the (almost) same name. It’s very actively developed. Quartz has an IJob interface with just one method, Execute, to implement.

 

But no out of the box support for multiple execution nodes (pooling or clustering), no adminstration UI that allows all job scheduling and configuration to be done outside of code, no monitoring and alerts,Insufficient mechanisms for dealing with errors/failures and recovery.

 

Based on the above observation it’s clear that we have quite some common issues faced on implementing the background task using above techniques:

  • An unhandled exception in a thread not associated with a request will take down the process.
  • If you run your site in a Web Farm, you could end up with multiple instances of your app that all attempt to run the same task at the same time.
  • The AppDomain your site runs in can go down for a number of reasons and take down your background task with it.

 

What is Hangfire?

Hangfire is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don’t want to put in your request processing pipeline. An easy way to perform background processing in .NET and .NET Core applications. No Windows Service or separate process required. Backed by persistent storage. Open and free for commercial use.

Hangfire allows you to kick off method calls outside of the request processing pipeline in a very easy, but reliable way. These method invocations are performed in a background thread and called background jobs.

Hangfire supports all kind of background tasks – short-running and long-running, CPU intensive and I/O intensive, one shot and recurrent. You don’t need to reinvent the wheel – it is ready to use.

Backed by Persistent Storage
Background jobs are very important part of an application and Hangfire ensures that any job is performed at least once. To persist background job information between application restarts, all the information is saved in your favorite persistent storage. Currently the following storages are supported:

  • SqlServer
  • Redis
  • PostgreSql
  • mongoDB

Below is the bird eye view of the hangfire architecture.

 

Why Hangfire?

  • Simple :

Easy to set up, easy to use. No Windows Service, no Windows Scheduler, no separate applications required. Background jobs are regular static or instance .NET methods with regular arguments – no base class or interface implementation required.

  • Persistent :

Background jobs are created in a persistent storage – SQL Server, Redis, PostgreSQL, MongoDB and others.You can safely restart your application and use Hangfire with ASP.NET without worrying about application pool recycles.

  • Transparent :

Built-in web interface allow you to see the whole picture of your background processing, as well as observe the state of each background job. Out of the box support for popular logging frameworks allows you to catch errors early with zero configuration.

  • Reliable :

Once a background job was created without any exception, Hangfire takes the responsibility to process it with the at least once semantics. You are free to throw unhandled exceptions or terminate your application – background jobs will be re-tried automatically.

  • Distributed :

Background method calls and their arguments are serialized and may overcome the process boundaries. You can use Hangfire on different machines to get more processing power with no configuration – synchronization is performed automatically.

  • Extensible :

Job filters allow you to add custom features to the background processing in a way similar to ASP.NET MVC action filters. Job storage access is fully abstracted and you can implement the support for your favorite storage. Dashboard supports modifications too.

  • Efficient :

Although the default installation uses SQL Server and polling technique to fetch jobs, you can leverage MSMQ or Redis extensions to reduce the processing latency to minimum.

  • Self-maintainable :

You don’t need to perform manual storage clean-up – Hangfire keeps it as clean as possible and removes old records automatically.

  • Open source :

Hangfire is open source software and is completely free for commercial use. It is licensed under LGPLv3 license. Fork the project and make contributions on GitHub!

 

Hangfire Setup

  • Installation

There are a couple of packages for Hangfire available on NuGet. To install Hangfire into your ASP.NET application with SQL Server storage, type the following command into the Package Manager Console window:

PM> Install-Package Hangfire

 

  • Configuration

After installing the package, add or update the OWIN Startup class with the following lines:

 

Authorization configuration required

By default only local access is permitted to the Hangfire Dashboard. Dashboard authorization must be configured in order to allow remote access.

Then open the Hangfire Dashboard to test your configuration. Please, build the project and open the following URL in a browser:

http://<your-site>/hangfire

  • Usage

Add a job, Hangfire handles different types of background jobs, and all of them are invoked on a separate execution context.

·         Fire-and-forget

These jobs are executed only once and almost immediately after they are fired.

·         Delayed

Delayed jobs are executed only once too, but not immediately – only after the specified time interval.

·         Recurring

Recurring jobs are fired many times on the specified CRON schedule.

·         Continuations

Continuations are executed when parent job has finished.

·         Batches

Batch is a group of background jobs created atomically.

·         Batch Continuations

Batch continuation is fired after all background jobs in a parent batch have finished.

 

 

·         Background Process

Use them when you need to run background processes continuously throughout the lifetime of your application.

 

 

Conclusion

Hangfire saves your jobs into persistent storage and processes them in a reliable way. It means that you can abort Hangfire worker threads, unload application domain or even terminate the process, and your jobs will be processed anyway. Hangfire flags your job as completed only when the last line of your code was performed, and knows that the job can fail before this last line. It contains different auto-retrying facilities, that can handle either storage errors or errors inside your code.

This is very important for generic hosting environment, such as IIS Server. They can contain different optimizations, timeouts and error-handling code (that may cause process termination) to prevent bad things to happen. If you are not using the reliable processing and auto-retrying, your job can be lost. And your end user may wait for its email, report, notification, etc. indefinitely.

 

About The Author

Leave a Reply

*