Task based concurrency in C#


I have recently switched over to a company that mainly works with C# and python.
One of my first assignments have been to implement a minimal scheduler I have been trying to wrap my head around task based concurrency in C#.
This was necessary since I decided upon a "worker -- producer" setup with a producer that fetches "tasks" from a database, and a collection of workers that executes the tasks independently. This provides the benefit of allowing us to scale the number of workers to fit the current systems cpu and memory resources.
This approach is not without limitations but most of them are basic thread safety concerns.
The main issue however have been my somewhat lacking experience using tasks, which have lead to this post.

Real stuff


Below I will try and explain the basics for my use case:

The goal is to run a specific function in a object as a new task and await the result. The command used will be on similar to the following:

public async Task<bool> Execute(int n)
{
    return n == 0;
}

And the basic caller function will be similar to the following:

public async void Run()
{
    while(true)
    {
        if(!task.Empty)
        {
            Command c = tasks.Next();
            // MAGIC PART
         }
     }
}

The command class in the example implements my Execute function and the "Run" function will run in a worker object.

My initial attempts at "magic" ended up with the task running synchronously with the calling thread blocking and awaiting the result. This behavior was close to the desired result, however there was no possibility to specify await time, or to do other processing on the worker thread.

//Initial Magic
Task<bool> task = c.Execute();

To achieve my desired result this function call had to be adapted, I ended up having to create a "runner" task that handled the waiting for the intended task.

var runner = Task.Factory.StartNew(() =>
{
  var runningTask = c.Execute();
  return runningTask;
});

This "runner" task can then be ignored until the worker is free to await the result. Or as in my case it can be used to create a timeout loop for the task.

while (waited < timeOut)
{
  if (runner.Wait(500) || runner.IsCompleted)
  {
    break;
  }
    Thread.Sleep(1);
    waited += 500;
}

This loop "pools" the task every 500 ms to see if it has completed, and if the timeout value is reached I am free to cancel the task.