Working with JSON in Python Flask

Working with JSON in Python Flask

With the advent of JavaScript based web technologies and frameworks like AngularJS, Node.js etc., knowing how work with JSON is a must. In this tutorial, we’ll see how to use JSON in Python Flask web application. JSON is a lightweight data format which is widely used across web applications for interchanging data across and within web applications. JSON means JavaScript Object Notation. From the official docs,

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition – December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

An example of a JSON object is :

var jsonObject = [{

        "FirstName": "Code",
        "Lastname": "Handbook"
    }, {
        "FirstName": "Ravi",
        "Lastname": "Shanker"
    }, {
        "FirstName": "Salman",
        "Lastname": "Khan"
    }, {
        "FirstName": "Ali",
        "Lastname": "Zafar"
    }

];

What we’ll learn in this tutorial

In this tutorial, we’ll see how to work with JSON in Python. For the sake of simplicity, we’ll be using Flask framework for creating a simple web application and see how to interchange JSON in Python from server side to client side. This tutorial assumes the user to have the basic knowledge of Python programming language and Flask framework.

Getting Started

Once you have Flask and Python installed in your system, create a file called app.py and paste the following code:

from flask import Flask
app = Flask(__name__)

@app.route("/getEmployeeList")
def getEmployeeList():
    return "Return Employee JSON data"

if __name__ == "__main__":
    app.run()

Try running the app.py file and you should be able to view the application running at http://localhost:5000/getData.

python app.py

Return JSON in Python

Generally in an application it’s easier to manipulate data fetched from database if it’s kept as objects. And whenever we need to transfer the data to client side, we simply convert the data to JSON and send it to the client side. What we’ll try to do here is create a class , say for example, a class for employee. Then we’ll try to create a list of employee class object and convert it into JSON and transfer it to the client side.

Let’s start by creating an employee class in employee.py file :

class Employee:
    def __init__(self,firstName,lastName):
        self.firstName = firstName
    self.lastName = lastName

Import the employee.py file in the app.py, so that we can create an employee object.

from employee import Employee

Inside the getEmployeeList method in app.py, create 5 instances of the employee class.

# Initialize a employee list
employeeList = []

# create a instances for filling up employee list
for i in range(0,5):
    employee = Employee('first name','last name')
    employeeList.append(employee)

As seen in the above code, we have created 5 instances of the employee class and inserted it into employeeList. Just to make it a bit more realistic, let’s replace the hard coded first name and last name with random names. Install package names for random name generator.

pip install names

Import the above names package in the app.py file.

import names

Replace the hard coded first name and last as shown:

# create a instances for filling up employee list
for i in range(0,5):
    employee = Employee(names.get_first_name(),names.get_last_name())
    employeeList.append(employee)

Define a method called toJSON in the employee.py which would return the JSON for an employee instance.

def toJSON(self):
    return {"Employees": {'firstName': self.firstName,
                 'lastName': self.lastName}}

Next we’ll iterate the employee list object in the app.py and convert each employee object to JSON and then dump the data using json.dumps. Import JSON in the app.py file and then add the following code to convert employeeList object to JSON data.

jsonStr = json.dumps([e.toJSON() for e in employeeList])

Here is the full code from the app.py file:

from flask import Flask,json
from employee import Employee
import names

app = Flask(__name__)

@app.route("/getEmployeeList")
def getEmployeeList():
    
    try:

        # Initialize a employee list
        employeeList = []

        # create a instances for filling up employee list
        for i in range(0,5):
            employee = Employee(names.get_first_name(),names.get_last_name())
            employeeList.append(employee)
    
    # convert to json data
        jsonStr = json.dumps([e.toJSON() for e in employeeList])

    except:
        print "error ", sys.exc_info()[0]

    return jsonStr

if __name__ == "__main__":
    app.run()

Try restarting the server and point your browser URL to http://localhost:5000/getEmployeeList and you should be able to view the JSON data in the web browser.

Convert Cursor Result Set to JSON

In the above section we saw how to convert a class instance or object list to JSON data. Now we’ll see how to convert a cursor result set to JSON data.

Assume that we are using the below code to fetch data from database :

resultSet = cursor.fetchall()

resultSet would be a list of items fetched from database. What we’ll do is iterate over the resultSet list and convert each item into a python dictionary with custom key.

empList = []
for emp in resultSet:
    empDict = {
        'Id': emp[0],
        'Name': emp[1],
        'Location': emp[2],
        'Address': emp[3]}
    empList.append(empDict)
return json.dumps(empList)

Wrapping It Up

In this tutorial, we saw how to work with JSON in python when writing web applications. Here I have dealt with two general cases where we need to convert the python class objects and result sets retrieved from database to JSON data. Have you worked with JSON data in python and if so, have you encountered any programming issues converting to JSON?

Do let us know you thoughts, suggestions and any corrections in the comments below.

Do have a look at other Python programming tutorials on CodeHandbook.

7 thoughts on “Working with JSON in Python Flask

  1. Hi,

    I tried this way, the output does come, but not as expected.
    This is how the JSON is outputted:

    “[{“Employees”: {“firstName”: “temp-name-1”}}, {“Employees”: {“firstName”: “temp-name-2″}}]”

    whereas I want the json as
    [{“Employees”: [{“firstName “: “temp-name-1”}, {“firstName”: “temp-name-2”}]}]

    How can this be achieved ?

    Please help.
    Thanks in advance.

      1. Hi,

        I tried your suggestion mentioned in the above URL.
        But, the problem is that it works only for a list defined in the code, like you did.
        It does not work for a list returned from MySQL DB with SQL Alchemy
        Here is the code which I am trying:

        empList = Employees.query.filter_by(user_id=current_user_id)

        jsonStr = json.dumps(empList)

        strToReturn = jsonify(Employees=jsonStr)
        return strToReturn

        And here is the error stack:

        Traceback (most recent call last):
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 2000, in __call__
        return self.wsgi_app(environ, start_response)
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1991, in wsgi_app
        response = self.make_response(self.handle_exception(e))
        File “C:Python27Libsite-packagesflask_restful__init__.py”, line 271, in error_router
        return original_handler(e)
        File “C:Python27Libsite-packagesflask_corsextension.py”, line 188, in wrapped_function
        return cors_after_request(app.make_response(f(*args, **kwargs)))
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1567, in handle_exception
        reraise(exc_type, exc_value, tb)
        File “C:Python27Libsite-packagesflask_restful__init__.py”, line 268, in error_router
        return self.handle_error(e)
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1988, in wsgi_app
        response = self.full_dispatch_request()
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1641, in full_dispatch_request
        rv = self.handle_user_exception(e)
        File “C:Python27Libsite-packagesflask_restful__init__.py”, line 271, in error_router
        return original_handler(e)
        File “C:Python27Libsite-packagesflask_corsextension.py”, line 188, in wrapped_function
        return cors_after_request(app.make_response(f(*args, **kwargs)))
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1544, in handle_user_exception
        reraise(exc_type, exc_value, tb)
        File “C:Python27Libsite-packagesflask_restful__init__.py”, line 268, in error_router
        return self.handle_error(e)
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1639, in full_dispatch_request
        rv = self.dispatch_request()
        File “X:dhprojproj_venvlibsite-packagesflaskapp.py”, line 1625, in dispatch_request
        return self.view_functions[rule.endpoint](**req.view_args)
        File “C:Python27Libsite-packagesflask_restful__init__.py”, line 477, in wrapper
        resp = resource(*args, **kwargs)
        File “X:dhprojproj_venvlibsite-packagesflaskviews.py”, line 84, in view
        return self.dispatch_request(*args, **kwargs)
        File “C:Python27Libsite-packagesflask_restful__init__.py”, line 587, in dispatch_request
        resp = meth(*args, **kwargs)
        File “C:Python27Libsite-packagesflask_jwt__init__.py”, line 177, in decorator
        return fn(*args, **kwargs)
        File “X:dhprojcode-baseprojappapihandler.py”, line 15, in get
        return get_emp_for_current_user()
        File “X:dhprojcode-baseprojappapifunctions.py”, line 16, in get_emp_for_current_user
        jsonStr = json.dumps(empList)
        File “X:dhprojproj_venvlibsite-packagesflaskjson.py”, line 126, in dumps
        rv = _json.dumps(obj, **kwargs)
        File “C:Python27Libjson__init__.py”, line 238, in dumps
        **kw).encode(obj)
        File “C:Python27Libjsonencoder.py”, line 203, in encode
        chunks = list(chunks)
        File “C:Python27Libjsonencoder.py”, line 436, in _iterencode
        o = _default(o)
        File “X:dhprojproj_venvlibsite-packagesflaskjson.py”, line 83, in default
        return _json.JSONEncoder.default(self, o)
        File “C:Python27Libjsonencoder.py”, line 178, in default
        raise TypeError(repr(o) + ” is not JSON serializable”)
        TypeError: is not JSON serializable

        Please help.

        Regards,
        ap ap

  2. On your explanation about “Convert Cursor Result Set to JSON”

    Your explanation assumes you know the fields in the JSON being read in. But I want to read ALL FIELDS, display them, and have them accessible so I can write them back out to a MongoDB database… which is just Json documents.

    How do I read in/ display without having to parse the JSON package to extract every dict field for your method? I assume it has to be doable?

    … using Python / Flask / MongoDB (so PRINT or PPRINT is not an option for display)

    YOUR CODE

    empList = []

    for emp in resultSet:

    empDict = {

    ‘Id’: emp[0],

    ‘Name’: emp[1],

    ‘Location’: emp[2],

    ‘Address’: emp[3]}

    empList.append(empDict)

    return json.dumps(empList)

    1. Ok . I’m not really sure if I understand you correctly. But let me if this is what you mean. You want to read a Json document from a mongo db database using python , modify it and write it back to mongo db ?

      1. Jay,

        Yes… but I don’t know the full structure of the documents before reading, so at the first pass, I’m just reading the entire document… displaying it (this is Flask / web, so no print statements) ..then writing it back out to MongoDB.

        Most of the examples I see assume you know what all the fields are beforehand, and that’s why I’m importing into MongoDB.. sources are from someone else’s web scrape or API outputs, so I don’t know all the fields available (although well enough structured).

        Some are more dicts like in structure…
        {
        {x,x,y},
        {x,x,x},
        {x,x,x}
        }

        and some are deeply nested [ ] and { } … (from an API)

        I’m going to pass some selected fields to an API, so I’ll be looking for some by name.

        So I do a

        client = pymongo.MongoClient(mongo_uri, ssl_ca_certs=’/xxx/tmp/cafile.pem’)

        mydb = client.dbname

        What I want to do is something like

        docs=mydb.collectname.find()
        mylist =[]
        for d in docs:
        mylist =mylist.append(mylist)
        return json.dumps(mydict)

        rather than something lik
        mylist = {‘report_id’:d[‘report_id’], ‘report_source’:d[‘report_source’]}

        but either way I’m getting [ ] or a mongodb.cursor.

        (I understand there are security concerns around jsonify or json.dumps of this type of json data, and that is in part why it’s harder to do what I thought was a simpler thing>

  3. Hi. I get the following AttributeError: ‘Employee’ object has no attribute ‘toJSON’
    If I change the code to make each employee a dict: {“FirstName”: employee.firstName, “LastName”: employee.lastName}, then append the dictionary to employeeList, making it simply a list of dict, and then finally change function containing the “e.toJSON()” method to:
    jsonStr = json.dumps([e for e in employeeList])
    … it seems to run fine.
    What am I missing?

Leave a Reply

Your email address will not be published. Required fields are marked *