Aggregate Formula - Time

Aggregate Formula - Time

I have a CRM table which has phone call start and end time.  I have created a Query table that also includes a Call Duration column (using Time_Diff). 

I am creating a summary report on Analytics and want to show the sum of the duration of inbound/outbound calls in the period specified in the user filter, broken down by CRM user.  When I create an aggregate formula though, time isn't a number format option.  If I choose decimal, it just shows 0 for each user. 

Is it possible to achieve the desired outcome and if so, how should I go about it?