About data source refresh

Klipfolio allows you to set a refresh queuing interval on a data source to automate the process of updating the data in a Klip. By doing this, your Klips display the most current information for key decision makers.

Data sources are automatically queued to refresh as soon as you log in to Klipfolio, and if their next scheduled refresh time is due or past due. At this time, a data source is sent to a global refresh queue.

The time it takes for a data source to refresh depends on:

  • the number of data sources already in the queue,
  • the time it takes for the source system to respond to the query defined in the Klipfolio data source, and
  • whether or not the refresh needs to be deferred in order to avoid reaching an API call limit, for an account, for a selected data source.

The Refresh Queue

A data source then continues to be added to the refresh queue depending on the interval set in the data source definition, and as long as there is some activity on the dashboard. For example, if you have a data source set to refresh every hour, as long as one person is active in the account, which might include viewing the dashboard, working in the Klip Editor, or viewing data sources in the Library, then all data sources for that organization are added to the global refresh queue when their next refresh time is reached.

Data sources are refreshed on a first-in-first-out order of priority. This ensures that resources are made available to active users. The only exception is deferred refreshes. When a refresh is deferred, passed the scheduled refresh time frame, the deferred refresh request is automatically added to the front of the queue.

Note for Partner accounts: For a client account to be recognized as active, a person must be signed into the client account, not the partner account. If no-one is signed in, the next person to sign in, will send data sources that have a next refresh time of before that login, to the global refresh queue.

The Deferred Queue

Because some APIs, like Google Analytics, impose a limit on the number of requests per profile, number of concurrent requests per profile, and the number of tokens per Google user account, Klipfolio monitors the number of API calls for each OAuth token and Google Analytics profile. This is most commonly seen when you have a large number of Google Analytics queries set to refresh in close succession. In this case, Klipfolio can defer a new refresh request from executing for a short period of time in order to prevent an account from violating an enforced limit. This typically has little impact on queue refresh times.

Embedded klips

Embedded Klips provide a simple way to expose your Klips on an external web site. Each time you view an embedded Klip on a web page, Klipfolio checks to see if the Klip has been rendered within the last three hours. If the Klip has been rendered in three hours or less, the cached Klip data is presented to the viewer. If the Klip has not been rendered, the Klip will now be rendered using the current cached data from your Dashboard. You must log in to your Dashboard in order for the data source cache to refresh.

Scheduled e-mail reports

When you schedule an e-mail report, Klipfolio automatically refreshes all data sources for the Klips contained in the email just before the scheduled email is due to be sent out.

Related links:


Connecting to data sources

  1. Klipfolio Data Source Integrations
  2. Creating data sources
  3. Best practices for uploading your data
  4. Updating data sources
  5. About data source refresh
  6. Install the latest Google Analytics KlipStart Dashboard
  7. NEW: Install the latest Google AdWords KlipStart (for MCC accounts only)
  8. Guide to Web Accessible (REST API) data sources
  9. Authentication method: OAuth Token Authentication for Web Accessible Data Source
  10. Why does Klipfolio need specific permissions to connect my service accounts?
  11. Authentication method: 2 Step Authentication
  12. Authentication method: Basic HTTP Authentication
  13. Authentication method: API Key Authentication
  14. Managing OAuth tokens for connected accounts
  15. Use date range parameters to modify data retrieval period
  16. NEW: Signed Authentication Builder (Hash Helper) (advanced)
  17. Locate a data source ID
  18. SQL Databases
  19. Use files from your computer as a data source
  20. Use Email Attachments as a data source
  21. Use Excel spreadsheets as a data source
  22. NEW: Use Adobe Analytics as a data source
  23. NEW: Use Alexa as a data source
  24. Use Asana as a data source
  25. NEW: Use Atlassian JIRA as a data source
  26. NEW: Use AWeber as a Data Source
  27. NEW: Use Basecamp as a data source
  28. Use Bitly as a data source
  29. Use Box as a data source
  30. NEW: Use Buffer as a data source
  31. NEW: Use CallRail as a data source
  32. Use Campaign Monitor as a data source
  33. NEW: Use Chargify as a data source
  34. Use Cloud Monitor as a data source
  35. Use comScore Digital Analytix as a data source
  36. NEW: Use Desk.com as a data source
  37. Use Dropbox as a data source
  38. Use Eloqua as a data source
  39. Use Facebook as a data source
  40. NEW: Use Facebook Ads as a data source
  41. NEW: Use FluidSurveys as a data source
  42. Use Freshdesk as a data source
  43. NEW: Use Fusebill as a data source
  44. Use FTP or SFTP as a data source
  45. NEW: Use Glassdoor as a data source
  46. Use Google+ as a data source
  47. NEW: Use Google AdWords as a data source
  48. Use Google Analytics as a data source
  49. Use Google Analytics to create a Multi-Channel Funnel (MCF) API data source
  50. NEW: Use Google Calendar as a data source
  51. Use Google Drive Apps as a data source
  52. Use Google Webmaster Tools as a data source
  53. NEW: Use Harvest as a Data Source
  54. NEW: Use GoSquared as a data source
  55. Use Heartbeat as a Data Source
  56. NEW: Use HubSpot as a data source
  57. NEW: Use iFormBuilder as a data source
  58. PREVIOUS: Use iFormBuilder as a data source
  59. Use Infusionsoft as a data source
  60. NEW: Use Insightly as a data source
  61. NEW: Use Instagram as a data source
  62. NEW: Use Intercom as a data source
  63. Use Intervals as a data source
  64. NEW: Use Lighthouse as a Data Source
  65. Use LinkedIn as a data source
  66. Use Localytics as a data source
  67. Use MailChimp as a data source
  68. NEW: Use Mixpanel as a data source
  69. NEW: Use Marketo as a data source
  70. NEW: Use Moz as a data source
  71. NEW: Use New Relic as a data source
  72. Use Nicereply as a data source
  73. Use Optimizely as a data source
  74. NEW: Use Pardot as a data source
  75. Use Pingdom as a data source
  76. NEW: Use Pipedrive as a data source
  77. Use Podio as a data source
  78. NEW: Use QuickBooks as a data source
  79. Use QuickBooks with the Email Attachment Connector
  80. NEW: Use Quintly as a data source
  81. Use Radian6 as a data source
  82. NEW: Use RSS news feeds as a data source
  83. Use Salesforce as a data source
  84. Use Searchmetrics as a data source
  85. Use Sendible as a data source
  86. NEW: Use Shopify as a data source
  87. Use ShoreTelSky as a data source
  88. NEW: Use Slack as a data source
  89. Use Smartsheet as a data source
  90. NEW: Use Stripe as a data source
  91. Use SurveyGizmo as a data source
  92. NEW: Use Swrve as a data source
  93. Use Toggl as a data source
  94. NEW: Use Trello as a data source
  95. NEW: Use Taleo as a data source
  96. NEW: Use Talkwalker as a data source
  97. NEW: Use Todoist as a data source
  98. Use Topsy as a data source
  99. Use Twitter as a data source
  100. Use UserVoice as a data source
  101. NEW: Use VKontakte as a data source
  102. Use Webtrends as a data source
  103. NEW: Use Xero as a data source
  104. Use XMLA as a data source
  105. NEW: Use Yahoo Finance as a data source
  106. Use YouTube Analytics as a data source
  107. Use Zendesk as a data source
  108. Use Zoho CRM as a data source
  109. Automate data upload using the data source API
  110. Use Unbounce as a data source
  111. Data examples

Feedback and Knowledge Base