Kaizen #192 - Implementing Custom Token Persistence in Python SDK

Kaizen #192 - Implementing Custom Token Persistence in Python SDK


Welcome back to another week of Kaizen!

Last week, we discussed how to implement Login with Zoho using OAuth 2.0 and saw how to bring it to life in a real-world application with the Zoho CRM Python SDK. We also discussed how Zylker Academy built a custom student portal powered by Zoho authentication.
In our sample project, we used the file-based persistence method, a simple setup where the access and refresh tokens are stored in a local file. While this method is great for getting started, it might not always fit your business requirements.
That is why our SDKs offer multiple ways to persist your tokens. 

This week, we will explore why token persistence matters for your app’s secure operation, and how to implement custom token persistence methods, including a practical example using SQLite.

Why does token persistence matter?

When a user logs in via OAuth, Zoho returns two tokens:
  • An access token (valid for one hour), used to access Zoho CRM data.
  • A refresh token, used to get a new access token when the current one expires.
If your app does not store these tokens properly, your users will be forced to log in again every time they make an API call. Or every time their access token gets expired. That is not inconvenient; it is a poor user experience.
When you use Zoho CRM SDKs, this is all handled for you behind the scenes. When you first authenticate with Zoho, the SDK stores your access and refresh tokens. Later, when a token expires, the SDK automatically uses the refresh token to get a new one. All you have to do is configure and initialize the SDK, and you are ready to start making API calls using the different methods offered by our SDKs! 

From the user’s perspective, it means:
  • They do not have to log in every time.
  • Their sessions are automatically renewed without interruption.
  • Token revocation can be done centrally.
From a developer’s perspective:
  • You can control how and where tokens are stored.
  • You have control to enforce policies like session timeouts or token cleanup.

Supported token persistence options

The Zoho CRM SDKs support three token persistence mechanisms:

File Persistence:

As we have already seen in last week's Kaizen, in this method, the tokens are stored in a local file of your choice. This can be configured while configuring and initializing the SDK. While this is simple and great for internal and local use, it might not always meet the needs of a growing business. For instance, if the file gets deleted or corrupted, you lose the tokens. It also poses a security risk, as storing tokens in files may expose them to unauthorised access if the file is not properly secured.

Database Persistence:

This stores tokens in a MySQL database, making it better suited for production environments. It is more robust and can handle larger-scale user management. 
Using this persistence method, you can only provide the following connection parameters - host, DB name, table name, username, password, and port number. 

Custom Persistence:

But what if neither of these options fits your needs? Maybe you are working in an environment without traditional storage like AWS's Secret Manager, or you prefer any other Database, or running a microservice in a container where local storage is more practical. That is where Custom Token Persistence comes in.

Custom Token Persistence

Custom persistence means you can implement your own logic for storing and retrieving OAuth tokens, instead of relying on the SDK’s default mechanism. To do this, you should create a class that implements the TokenStore interface and override a standard set of methods, each handling a specific part of the token lifecycle.

Here’s what your custom class must implement:
Method
Purpose
Return Type
find_token(self, token)
Given a token, return a full Token (OAuthToken) object from storage. Used before making any CRM API call.
Token(OAuthToken) object
save_token(self, token)
Called right after Zoho returns a new access/refresh token. Your implementation must persist it.
None
delete_token(self, id)
Delete a specific token using its unique ID.
None
get_tokens(self)
Return all stored tokens.
A list of Token(OAuthToken) objects
delete_tokens()
Delete all stored tokens. Useful during cleanup or logout.
None
find_token_by_id(id)
Retrieve a token by its unique identifier.
Token(OAuthToken) object

The token object is an instance of OAuthToken. The SDK will invoke these methods automatically as part of its flow. You just have to focus on where and how to store the tokens. With this, you can persist tokens to any storage as long as your class handles these methods correctly.

Understanding the token object

Before we dive deeper into custom token persistence, let's clarify what this token (OAuthToken) object is and how you should work with it.

The token object is an instance of OAuthToken.  This class bundles all the credentials and details the SDK needs to authenticate your API requests. Here’s what it holds:
  • access_token
  • refresh_token
  • client_id 
  • client_secret
  • redirect_url
  • expires_in
  • user_signature
  • id
  • api_domain

Implementing Custom Token Persistence with SQLite

Now that we've covered the basics of token persistence and how Zoho SDK supports custom stores, let’s dive into a practical, real-world example using SQLite as the backend for storing tokens.
SQLite is a lightweight, file-based database engine. It is perfect when you want a persistent store without the complexity of a full database server.

The CustomStoreSQLite Class

This class implements all six required methods of the TokenStore interface using SQLite as the backend. 

1. Initialization and Table Setup

When you create a CustomStoreSQLite object, it immediately checks if the token table exists in the SQLite database file zohooauth.db. If the DB or the table is missing, its __init__() method creates one with all the necessary columns to store token details like id, user_name, client_id, client_secret, refresh_token, access_token, grant_token, expiry_time, redirect_url and api_domain.


 def __init__(self):
        """
        Initializes the SQLite database and sets up the oauthtoken table if needed.
        """
        self.db_name = 'zohooauth.db'
        if not self.check_table_exists():
            connection = sqlite3.connect(self.db_name)
            cursor = connection.cursor()
            cursor.execute("CREATE TABLE  oauthtoken (id varchar(10) NOT NULL,user_name varchar(255), client_id "
                           "varchar(255), client_secret varchar(255), refresh_token varchar(255), access_token "
                           "varchar(255), grant_token varchar(255), expiry_time varchar(20), redirect_url varchar("
                           "255), api_domain varchar(255), primary key (id))")
 cursor.close()

This means the first time your app runs, it sets up its own database schema automatically.

2. Saving a Token - save_token(self, token)

Purpose:
This method is called every time Zoho returns a new token, whether after a login or a token refresh. Your implementation is responsible for safely persisting this token, typically by upserting (inserting or updating) a row in your database that uniquely identifies the token’s user and client combination.

Expected behaviour: 
The method must store the token in your custom database or storage system.
  • If a matching token already exists (based on user, refresh token, or client credentials), it should be updated.
  • If no match exists, a new entry must be created.
  • Tokens should not be duplicated. Multiple users should be managed separately.
Input Parameters: An instance of Token(OAuthToken) class containing details like access token, refresh token, user signature, client ID/secret, etc.

Return value: None. But must raise exceptions on failure.

Sample Implementation using SQLite:
Here is the logic used in the implementation of save_token() method:
  • If the user name is available, use it to update the token.
  • If no user name but the access token is available in the table, update by the access token.
  • If there is a refresh or grant token with the same client credentials, then update accordingly.
  • If none of these match, insert as a new row.

def save_token(self, token):
        if not isinstance(token, OAuthToken):
            return
        cursor = None
        connection = None
        try:
            connection = sqlite3.connect(self.db_name)
            oauth_token = token
            query = "update oauthtoken set "
            if oauth_token.get_user_signature() is not None:
                name = oauth_token.get_user_signature().get_name()
                if name is not None and len(name) > 0:
                    query = query + self.set_token(oauth_token) + " where user_name='" + name + "'"
            elif oauth_token.get_access_token() is not None and len(oauth_token.get_access_token()) > 0 and \
                    self.are_all_objects_null([oauth_token.get_client_id(), oauth_token.get_client_secret()]):
                query = query + self.set_token(
                    oauth_token) + " where access_token='" + oauth_token.get_access_token() + "'"
            elif ((oauth_token.get_refresh_token() is not None and len(oauth_token.get_refresh_token()) > 0) or
                  (oauth_token.get_grant_token() is not None and len(
                      oauth_token.get_grant_token()) > 0)) and oauth_token.get_client_id() is not None \
                    and oauth_token.get_client_secret() is not None:
                if oauth_token.get_grant_token() is not None and len(oauth_token.get_grant_token()) > 0:
                    query = query + self.set_token(
                        oauth_token) + " where grant_token='" + oauth_token.get_grant_token() + "'"
                elif oauth_token.get_refresh_token() is not None and len(oauth_token.get_refresh_token()) > 0:
                    query = query + self.set_token(
                        oauth_token) + " where refresh_token='" + oauth_token.get_refresh_token() + "'"
            query = query + " limit 1"
            try:
                cursor = connection.cursor()
                cursor.execute(query)
                if cursor.rowcount <= 0:
                    if oauth_token.get_id() is not None or oauth_token.get_user_signature() is not None:
                        if oauth_token.get_refresh_token() is None and oauth_token.get_grant_token() is None \
                                and oauth_token.get_access_token() is None:
                            raise SDKException(Constants.TOKEN_STORE, Constants.GET_TOKEN_DB_ERROR1)
                    if oauth_token.get_id() is None:
                        newId = str(self.generate_id())
                        oauth_token.set_id(newId)
                    query = "insert into oauthtoken (id,user_name,client_id,client_secret,refresh_token,access_token," \
                            "grant_token,expiry_time,redirect_url,api_domain) values (?,?,?,?,?,?,?,?,?,?);"
                    val = (token.get_id(),
                           token.get_user_signature().get_name() if token.get_user_signature() is not None else None,
                           token.get_client_id(), token.get_client_secret(), token.get_refresh_token(),
                           token.get_access_token(), token.get_grant_token(), token.get_expires_in(),
                           token.get_redirect_url(), token.get_api_domain())
                    cursor.execute(query, val)
            except Error as e:
                raise e
            finally:
                connection.commit()
                cursor.close() if cursor is not None else None
                connection.close() if connection is not None else None
        except Exception as ex:
 raise SDKException(Constants.TOKEN_STORE, Constants.SAVE_TOKEN_DB_ERROR, cause=ex)

3: Fetching a Token - find_token(self, token)

Purpose:
This is the method the SDK calls whenever it needs to make an API call on behalf of a user, but has only partial token information.
Depending on the token flow - Grant Token, Refresh Token, Access Token, or ID-based - only a specific token or ID may be provided during the API call. In such cases, find_token(self, token) method locates and return the complete OAuthToken object from storage if a matching one exists. If no matching token exists in the storage, this method will return None, and the SDK will proceed to generate a new token with the provided details and save it using the save_token(self, token) method. 

Expected behavior:
  • Based on the available details in the input token (user name, access token, refresh or grant token), this method should query storage and return a complete token object.
  • If no match is found, it should return None.
Input Parameters: A partially filled Token(OAuthToken) object.

Return value: A fully populated Token object if found, or None.

Sample Implementation using SQLite:
The find_token(self, token) method implementation does the following:
  • Dynamically builds a WHERE clause based on available attributes.
  • Queries the database for a matching record.
  • Fetches the matching record, if any, and populates the Token object with the full set of stored values (access token, refresh token, expiry time, etc.).
  • Returns the Token object if a matching record is found, or return None.
Without this method, your app wouldn’t know which token to use during API calls. For example, consider the case when a user reopens your app after hours. You have their refresh token stored. The SDK calls find_token(self, token) to get the full token and proceeds without requiring a fresh login.

def find_token(self, token):
        cursor = None
        connection = None
        try:
            connection = sqlite3.connect(self.db_name)
            if isinstance(token, OAuthToken):
                oauth_token = token
                query = "select * from oauthtoken"
                if oauth_token.get_user_signature() is not None:
                    name = oauth_token.get_user_signature().get_name()
                    if name is not None and len(name) > 0:
                        query = query + " where user_name='" + name + "'"
                elif oauth_token.get_access_token() is not None and self.are_all_objects_null(
                        [oauth_token.get_client_id(), oauth_token.get_client_secret()]):
                    query = query + " where access_token='" + oauth_token.get_access_token() + "'"
                elif oauth_token.get_refresh_token() is not None or oauth_token.get_grant_token() is not None and \
                        oauth_token.get_client_id() is not None and oauth_token.get_client_secret() is not None:
                    if oauth_token.get_grant_token() is not None and len(oauth_token.get_grant_token()) > 0:
                        query = query + " where grant_token='" + oauth_token.get_grant_token() + "'"
                    elif oauth_token.get_refresh_token() is not None and len(oauth_token.get_refresh_token()) > 0:
                        query = query + " where refresh_token='" + oauth_token.get_refresh_token() + "'"
                query = query + " limit 1"
                cursor = connection.cursor()
                cursor.execute(query)
                result = cursor.fetchone()
                if result is None:
                    return None
                self.set_merge_data(oauth_token, result)
        except Exception as ex:
            raise SDKException(Constants.TOKEN_STORE, Constants.GET_TOKEN_DB_ERROR1, cause=ex)
        finally:
            cursor.close() if cursor is not None else None
            connection.close() if connection is not None else None
        return token

4: Deleting a Token - delete_token(self, id)

Purpose:
Delete a specific token record from storage based on a unique token ID. It is commonly used when a user logs out or an admin revokes access for a user.

Expected behaviour:
  • Locate the token record by its unique ID.
  • Delete the corresponding record from storage.
Input Parameters: The token ID to be deleted.

Return values: None

Sample Implementation using SQLite:

def delete_token(self, id):
        cursor = None
        try:
            connection = sqlite3.connect(self.db_name)
            try:
                cursor = connection.cursor()
                query = "delete from oauthtoken where id= " + id + ";"
                cursor.execute(query)
                connection.commit()
            except Error as ex:
                raise ex
            finally:
                cursor.close() if cursor is not None else None
                connection.close() if connection is not None else None
        except Error as ex:
            raise SDKException(code=Constants.TOKEN_STORE, message=Constants.DELETE_TOKEN_DB_ERROR, cause=ex)

5: Deleting All Tokens - delete_tokens(self)

Purpose: Delete all tokens from storage, typically used for global logout or cleanup scenarios.

Expected behaviour: Remove all token records from storage in a single operation.

Input Parameters: None

Return Values: None

Sample Implementation using SQLite:

def delete_tokens(self):
        cursor = None
        try:
            connection = sqlite3.connect(self.db_name)
            try:
                cursor = connection.cursor()
                query = "delete from oauthtoken;"
                cursor.execute(query)
                self.connection.commit()
            except Error as ex:
                raise ex
            finally:
                cursor.close() if cursor is not None else None
                connection.close() if connection is not None else None
        except Error as ex:
            raise SDKException(code=Constants.TOKEN_STORE, message=Constants.DELETE_TOKENS_DB_ERROR, cause=ex)

6: Fetch all tokens - get_tokens(self)

Purpose: Retrieve all currently stored tokens.

Expected behaviour:
  • Query storage for all token records.
  • Construct and return a list of token objects 
Input Parameters: None

Return Value: A list of Token objects representing all stored tokens.

Sample Implementation using SQLite:


def get_tokens(self):
        cursor = None
        try:
            connection = sqlite3.connect(self.db_name)
            tokens = []
            try:
                cursor = connection.cursor()
                query = "select * from oauthtoken;"
                cursor.execute(query)
                results = cursor.fetchall()
                for result in results:
                    oauth_token = object.__new__(OAuthToken)
                    self.set_oauth_token(oauth_token)
                    self.set_merge_data(oauth_token, result)
                    tokens.append(oauth_token)
                return tokens
            except Error as ex:
                raise ex
            finally:
                cursor.close() if cursor is not None else None
                connection.close() if connection is not None else None
        except Error as ex:
            raise SDKException(code=Constants.TOKEN_STORE, message=Constants.GET_TOKENS_DB_ERROR, cause=ex)

7. Finding a Token by ID - find_token_by_id(self, id)

Purpose: Retrieve a specific token by its unique id.

Expected behaviour:
  • Search storage for a token with the given ID.
  • If found, return the complete token object; if not, return None.
Input Parameters: The unique identifier of the token (id)

Return Values: Returns a fully populated Token(OAuthToken) object if found; otherwise, returns None.

Sample Implementation using SQLite:
This method should follows a similar pattern to find_token, but use the unique id as the search key.

 def find_token_by_id(self, id):
        cursor = None
        try:
            connection = sqlite3.connect(self.db_name)
            try:
                query = "select * from oauthtoken where id='" + id + "'"
                oauth_token = object.__new__(OAuthToken)
                self.set_oauth_token(oauth_token)
                cursor = connection.cursor()
                cursor.execute(query)
                results = cursor.fetchall()
                if results is None or len(results) <= 0:
                    raise SDKException(Constants.TOKEN_STORE, Constants.GET_TOKEN_BY_ID_DB_ERROR)
                for result in results:
                    self.set_merge_data(oauth_token, result)
                    return oauth_token
            except Error as ex:
                raise ex
            finally:
                cursor.close() if cursor is not None else None
                connection.close() if connection is not None else None
        except Error as ex:
            raise SDKException(code=Constants.TOKEN_STORE, message=Constants.GET_TOKEN_BY_ID_DB_ERROR, cause=ex)


Please find the complete custom_store_sqlite.py file here.

How to use this in your project

To start using this custom token persistence class in your own Python project, follow these steps:
  • Download the custom_store_sqlite.py and place this inside your project directory.
  • Import the class in the script where you initialize the SDK. In our sample project, this is the record.py file.
    from store.custom_store_sqlite import CustomStoreSQLite
  • In the SDK configuration, use the CustomStoreSQLite method instead of the FireStore method:

def init(self, client_id, code, location, redirect_url):
        environment = DataCenter.get(location)
        client_secret = "17565609051856218813123b9a98de52c301722b7d"
        logger = Logger.get_instance(level=Logger.Levels.INFO,
                                     file_path="./logs.txt")
        store = CustomStoreSQLite()
        token = OAuthToken(client_id=client_id,
                           client_secret=client_secret,
                           grant_token=code,
                           redirect_url=redirect_url)
        Initializer.initialize(environment=environment,
                               token=token,
                               logger=logger,
 store=store)

That’s it! With this, all token operations (save, fetch, delete) will be routed through your custom store backed by SQLite.


The above video demonstrates this is in action. You can see what the database looks like when populated. 

More Custom Persistence Implementations

The advantage of using Zoho CRM SDKs is that it doesn't box you in. You are free to implement token persistence in a way that fits your business logic, team expertise, or project requirements. Whether you prefer SQLite, NoSQL, or something entirely different, the SDK gives you full control through the TokenStore interface.

In the SQLite example above, we walked through how to implement a custom store using a persistent file-based database. You need to implement all the methods as explained in the previous section, no matter where you decide to persist your tokens. 

To make things easier, we have included two additional reference implementations:
  • An in-memory store, where tokens are stored in a dictionary
  • A list-based store, which keeps token records as simple lists
Each one fully implements the required methods of the TokenStore interface.

SQLite In-Memory DB

This implementation uses SQLite's in-memory mode (using ":memory:") to store tokens in RAM. Here, we have implemented all the required methods from the TokenStore interface: find_token(), save_token(), delete_token(), get_tokens(), delete_tokens() and find_token_by_id().

Please find the custom_store_in_memory.py file here.

List-Based Persistence Using Simple Lists

The second reference implementation is a list-based token store that keeps token records in an in-memory Python list of lists. Each inner list represents a token’s attributes, such as ID, user signature, client ID, access token, refresh token, and so on.
This custom store fully implements all required methods from the TokenStore interface.

Please find the custom_store_list.py file here.

We hope this was useful and gives you enough info to build your own token persistence methods tailored to your needs. We used Python SDK here, but you can apply the same logic with any of our other SDKs. It is all the same logic, just different programming languages. Just remember to implement the required methods exactly as expected by the SDK, as explained here.

Give it a try, and please let us know how it goes or if you hit any bumps!  Comment below, or send an email to support@zohocrm.com. We will be waiting to hear from you!

Happy coding!

We are excited to be approaching the 200th post in our Kaizen series! As we get closer to this milestone, we would love to hear from you. Have questions, suggestions, or topics you would like us to cover in our future Kaizen posts? Your feedback helps us make the series even better.
 
Please take a moment to share your thoughts with us using this form - we'd really appreciate it!

  Previous Kaizen: Kaizen #191 - Implementing "Login with Zoho" using Python SDK |  Kaizen Directory                    


    • Sticky Posts

    • Kaizen #217 - Actions APIs : Tasks

      Welcome to another week of Kaizen! In last week's post we discussed Email Notifications APIs which act as the link between your Workflow automations and you. We have discussed how Zylker Cloud Services uses Email Notifications API in their custom dashboard.

    • Kaizen #216 - Actions APIs : Email Notifications

      Welcome to another week of Kaizen! For the last three weeks, we have been discussing Zylker's workflows. We successfully updated a dormant workflow, built a new one from the ground up and more. But our work is not finished—these automated processes are

    • Kaizen #152 - Client Script Support for the new Canvas Record Forms

      Hello everyone! Have you ever wanted to trigger actions on click of a canvas button, icon, or text mandatory forms in Create/Edit and Clone Pages? Have you ever wanted to control how elements behave on the new Canvas Record Forms? This can be achieved

    • Kaizen #142: How to Navigate to Another Page in Zoho CRM using Client Script

      Hello everyone! Welcome back to another exciting Kaizen post. In this post, let us see how you can you navigate to different Pages using Client Script. In this Kaizen post, Need to Navigate to different Pages Client Script ZDKs related to navigation A.

    • Kaizen #210 - Answering your Questions | Event Management System using ZDK CLI

      Hello Everyone, Welcome back to yet another post in the Kaizen Series! As you already may know, for the Kaizen #200 milestone, we asked for your feedback and many of you suggested topics for us to discuss. We have been writing on these topics over the