My manager assigned me a project whereby I have been using jQuery Calendar plugin to display calendar data stored across various tables in our SQL database.
It's just a jQuery plugin that takes static json data and renders it on a calendar. I had to integrate it with .net and our SQL database in such a way that the calendar could render the data from the SQL database (Microsoft SQL.)
Initially we put this together in such a way that we fetched all the data from the SQL server and then used .Net to construct the json and then pass it onto the jQuery calendar plugin.
Although in principle this worked well, it was extremely slow and IIS was often timing out. Not to mention, every time any of us wanted to view the calendar we had to wait around 3mins since the number of entries is approaching 3000.
The queries were quite complex, they're using on the fly Dateadd and DateDiff functions and alsorts of manners of operations. Execution time on the SQL server alone was around 90 seconds for the query. In total query size was around 160kb.
We then split the query into 3 parts (for different departments), but the amount of time we have to wait for the calendar to draw is still over a minute.
Here is an example of just one of the queries but there are over 100 of these per department
CREATE TABLE #AnnualLastMonImportantCustomDate(
Title varchar(550) COLLATE Latin1_General_CI_AS NULL,
AllocatedDate varchar(550) COLLATE Latin1_General_CI_AS NULL,
EndDateTime varchar(550) COLLATE Latin1_General_CI_AS NULL,
url varchar(550) COLLATE Latin1_General_CI_AS NULL,
width varchar(10) COLLATE Latin1_General_CI_AS NULL,
height varchar(550) COLLATE Latin1_General_CI_AS NULL,
AllDay varchar(550) COLLATE Latin1_General_CI_AS NULL,
description varchar(550) COLLATE Latin1_General_CI_AS NULL,
color varchar(550) COLLATE Latin1_General_CI_AS NULL,
textColor varchar(550) COLLATE Latin1_General_CI_AS NULL
)
DECLARE db_cursor CURSOR FOR SELECT AlertDate FROM xsCRMAlerts
WHERE AlertType='InternalImportantDate'
-- cursor is the results row when table goes through fetch process
SET @MyTableName='xsCRMAlerts'
OPEN db_cursor -- opens the table and stores id, which is the primary key in the table
FETCH NEXT FROM db_cursor INTO @MyTableName -- @MyTableName in this case is the result row.
WHILE @@FETCH_STATUS = 0 -- 0 is for success -1 is for too many results -2 is for the row fetched is missing
BEGIN
-- Below between begin and end the statement is linked to a function, which gives the dates tabled based on a start date. This table is then cross joined to produce desired result.
SET @startDate = @MyTableName -- we can set the start date to all the data we recieved because we have only asked for one field in our @MyTableName query when db_cursor was being drawn
INSERT INTO #AnnualLastMonImportantCustomDate
SELECT
'Important Date : ' + [Title] as 'Title',
dr.date as 'AllocatedDate',
dr.date as 'EndDateTime' ,
'xsCRM_Dates_Edit.aspx?id=' + cast(id as varchar) as 'url' ,
'515px' as 'width',
'410px' as 'height',
'true' as 'allDay',
'Important date' as 'description', /* This is a static entry and will not show on the calendar. Used when redering object*/
'yellow' as 'color',
'black' as 'textColor'
FROM [DelphiDude].[dbo].[xsCRMAlerts]
cross JOIN
dateTable(
DATEADD(yy,DATEDIFF(yy,0,GETDATE()),0)
,
DateAdd(yy,1,DATEADD(ms,-3,DATEADD(yy,0,DATEADD(yy,DATEDIFF(yy,0,GETDATE())+1,0))))
) dr -- You can specify intervals by calling DateTable_Month, DateTable_Quarter, DateTable_BiAnnual and DateTable_Annual
WHERE
(AlertType='InternalImportantDate') and
(occurring='765') and
(Datepart(m,date) = 12) and
(Datepart(day,date) > 24) and
(Datepart(dw,date) = 2) and
(Datepart(year,date) = (Datepart(year,getDate()) + 1))
FETCH NEXT FROM db_cursor INTO @MyTableName -- gets the next record from the table
END
CLOSE db_cursor
DEALLOCATE db_cursor
We really do need these queries.
We've now thought about limiting the result set to just the prev and next 30 days.
But each time we optimise a query, I then (even if its just using find and replace) have to replicate that change across 100 queries per module.
Is there a way we can optimise these queries and speed up the execution and calendar rendering time thats definitive and improves it by a long shot? And is there a way that I can apply the changes in such a way that it replicates across each of the queries?
I suggested using caching, db caching and object caching to my boss but he said the data would be changing often and data from here needs to be passed onto other modules and therefore if it is cached could be inaccurate. I dont have enough experience to contest what he was saying.
Any advice anyone?