Django Rest Framework: Serialize data from nested json fields to plain object

Question

I want to serialize non-flat structure to a one flat object. Here's an example of an API call I receive (I cannot control it unfortunately):

{
"webhookEvent": "jira:issue_updated",
"user": {
        "id": 2434,
        "name": "Ben",
    },
"issue": {
      "id": "33062",
      "key": "jira-project-key-111",
      "fields": {
          "summary": "The week ahead",
      },
"changelog": {
    "id": "219580",
    "items": [{
         "field": "status",
         "fieldtype": "jira",
         "from": "10127",
         "fromString": "Submitted",
         "to": "10128",
         "toString": "Staged"
    }]
},
"timestamp": 1423234723378
}

I'd like to serialize it to the models like these:

class Issue(models.Model):
    jira_id = models.IntegerField()
    jira_id = models.CharField()
    summary = models.CharField()

class Change(models.Model):
    issue = models.ForeignKey(Issue)
    timestamp = models.DataTimeField()

As you can see, model Issue's field summary is located on the same object as id and key unlike in JSON data.

My Serializer are next:

    class ChangeSerializer(serializers.ModelSerializer):
        """Receives complex data from jira and converts into objects."""

        issue = JiraIssueSerializer()
        timestamp = TimestampField(source='created_at')

        class Meta:
            model = Change
            fields = ('issue', 'timestamp')

        def create(self, validated_data):
            super(serializers.ModelSerializer, self).create(validated_data=validated_data)
            jira_issue = JiraIssueSerializer(data=validated_data)
            issue = Issue.objects.get(jira_issue)
            self.created_at = datetime.utcnow()
            change = Change(**validated_data)
            return change



    class JiraIssueSerializer(serializers.ModelSerializer):
        """Issue serializer."""
        id = serializers.IntegerField(source='jira_id')
        key = serializers.CharField(source='jira_key')
        summary = serializers.CharField()   ### I want field to work!
        # fields = serializers.DictField(child=serializers.CharField())

        class Meta:
            model = Issue
            fields = ('id', 'key',
               'summary',
            )

        def to_internal_value(self, data):
           # ret = super(serializers.ModelSerializer,   self).to_internal_value(data)
           ret = {}
           # ret = super().to_internal_value(data)
           ret['jira_id'] = data.get('id', None)
           ret['jira_key'] = data.get('key', None)
           jira_issue_fields_data = data.get('fields')
           if jira_issue_fields_data or 1:
               summary = jira_issue_fields_data.get('summary', None)
               ret.update(summary=summary)
            print('to_internal_value', ret)
            return ret

         def to_representation(self, instance):
            ret = {}
            ret = super().to_representation(instance)
            fields = {}
            fields['summary'] = instance.summary
            ret.update(fields=fields)
            print(ret)
            return ret

I works well for fields in issue object in JSON. But how can I add to the JiraIssueSerializer some fields like summary? They are not direct fields of issue object, but located in substrcucture fields. I see these ways:

Make one more Model Fields to keep them, but it's ridiculous. I do not need it and my API strictly depends on the foreign structure.
Make some .to_internal_fields() convertors. But in this case I have to manually validate and prepare all the fields in my Issue and repeat myself several times. Also If field summary is not enlisted in Serializer, latter cannot use it or validation fails.
Add DictField to Serializer (commented in code) and take data from it. Is it good? How to validate it? Again, I have n clean structure of code.

Next I'd like to parse and save changelog data.

How to deal better with such structures?

Thank you!

Eugene Lisitsky · Accepted Answer · 2016-05-23 16:45:58Z

Finally, solution was found in tests of django-rest-framework.
https://github.com/tomchristie/django-rest-framework/blob/master/tests/test_serializer.py#L149

You may easily define nested serializers which will act as a containers and extract data to your plain object. Like so:

    class NestedSerializer1(serializers.Serializer):
        a = serializers.IntegerField()
        b = serializers.IntegerField()

    class NestedSerializer2(serializers.Serializer):
        c = serializers.IntegerField()
        d = serializers.IntegerField()

    class TestSerializer(serializers.Serializer):
        nested1 = NestedSerializer1(source='*')
        nested2 = NestedSerializer2(source='*')

    data = {
        'nested1': {'a': 1, 'b': 2},
        'nested2': {'c': 3, 'd': 4}
     }

     serializer = TestSerializer(data=self.data)
     assert serializer.is_valid()

     assert serializer.validated_data == {
        'a': 1, 
        'b': 2,
        'c': 3, 
        'd': 4
    }

could you explain more about how did you solve this problem? Thanks!
@TonyWang, I've created 3 serializers: top level one TestSerializer, and 2 nested NestedSerializer1 and NestedSerializer2. Then in top level I set source="*", this way all nested fields will be propagated to parent serializer. So you may have a dict {"a":1, "b":2, "c":3,"d":4} that will be serialized to json object with 2 fields nested1 and nested2. First contains fields a and b. Second - c and d. With propriate values. Result: {'nested1': {'a': 1, 'b': 2}, 'nested2': {'c': 3, 'd': 4}}

djq · Accepted Answer · 2016-05-19 18:34:15Z

I would suggest you create your own custom serializer to receive the data. You can do this like so:

from rest_framework import serializers

class MySerializer(serializers.Serializer):
    """
    Custom serializer
    """
    id = serializers.IntegerField(read_only=True)
    name = serializers.CharField()


    def create(self, validated_data):
        """Create a new object"""
        validated_data['custom_value'] = 0 # you can manipulate and restructure data here if you wish
        return MyModel.objects.create(**validated_data)

You can then manipulate the data as you wish in the create() function. You could also create nested custom serializers to parse this data.

I've found how to circumvent it. And it's a standard planned way!

Danny Vu · Accepted Answer · 2019-09-10 07:57:51Z

Documentation has a good part of dealing with nesting in serialisation.

Basically, you want to create a separate class that has the nested values as follows:

class UserSerializer(serializers.Serializer):
    email = serializers.EmailField()
    username = serializers.CharField(max_length=100)

class CommentSerializer(serializers.Serializer):
    user = UserSerializer()
    content = serializers.CharField(max_length=200)
    created = serializers.DateTimeField()

Collectives™ on Stack Overflow

Django Rest Framework: Serialize data from nested json fields to plain object

3 Answers 3

2 Comments

2 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

2 Comments

Comments

Linked

Related