Convert JSON array API response to a struct

Question

Using Go, I am hoping to query an API endpoint and output the results to a Gorm SQLite DB. This has been done before (here) but I need to write the code myself.

The API endpoint returns a JSON array, and for each trade in the array, I would like to place it into a struct, then add that as row in the SQLite database.

The struct is defined below:

type Trade struct {
    TradeID      int64  `json:"id"`
    Price        string `json:"price"`
    Qty          string `json:"qty"`
    QuoteQty     string `json:"quoteQty"`
    Time         int64  `json:"time"`
    IsBuyerMaker bool   `json:"isBuyerMaker"`
    IsBestMatch  bool   `json:"isBestMatch"`
}

These types may seem weird but was determined by using PowerShell with the below code:

PS C:\Git> $body = @{"symbol" = "ETHEUR";"limit" = 1000}
PS C:\Git> $response = Invoke-RestMethod https://api.binance.com/api/v3/trades -Body $body
PS C:\Git> $response[0] | gm

   TypeName: System.Management.Automation.PSCustomObject

Name         MemberType   Definition
----         ----------   ----------
Equals       Method       bool Equals(System.Object obj)
GetHashCode  Method       int GetHashCode()
GetType      Method       type GetType()
ToString     Method       string ToString()
id           NoteProperty long id=21731777
isBestMatch  NoteProperty bool isBestMatch=True
isBuyerMaker NoteProperty bool isBuyerMaker=True
price        NoteProperty string price=3539.03000000
qty          NoteProperty string qty=0.28600000
quoteQty     NoteProperty string quoteQty=1012.16258000
time         NoteProperty long time=1620822731248

So, the Go function I have so far is as follows:


func getTrade(symbol string, limit int) {
    uri := fmt.Sprintf("https://api.binance.com/api/v3/trades?symbol=%v&limit=%v", symbol, limit)

    db, err := gorm.Open(sqlite.Open("test.db"), &gorm.Config{})
    if err != nil {
        panic("failed to connect database")
    }

    // Create the fields in the database based on the Trade Struct
    db.AutoMigrate(&Trade{})

    // Query Binance API endpoint
    response, err := http.Get(uri)

    // Log if an error occurred
    if err != nil {
        log.Fatalln(err)
    }

    // Defer closing object
    defer response.Body.Close()

    // Create trade struct
    trade := Trade{}

From here I have attempted but failed a few various methods of converting the response to a struct. These methods include:

// Unmarshal bytes to data struct
    _ = json.Unmarshal([]byte(responseData), &trade)

var myClient = &http.Client{Timeout: 10 * time.Second}

func getJson(url string, target interface{}) error {
    response, err := myClient.Get(url)
    if err != nil {
        return err
    }
    defer response.Body.Close()

    return json.NewDecoder(response.Body).Decode(target)
}

// Iterate through trades returned (this doesn't work)
    for _, trade := range responseData {
        tradeVariable := Trade{}
        tradeVariable.TradeID = trade.id
        fmt.Println(tradeVariable.TradeID)
    }

I'm feeling really stuck and like I'm missing something to do with marshalling, can someone please help me out?

For json arrays that contain objects you should use a Go slice of structs, not just a struct. i.e. trades := []Trade{} and pass &trades to json.Unmarshal and make sure NOT to discard the error with _ but instead properly check it and handle it. If there's something wrong with the struct's field types the error returned by json.Unmarshal will tell you very clearly. — mkopriva
– mkopriva, Commented May 12, 2021 at 12:57

bergwald · Accepted Answer · 2021-05-12 13:40:39Z

Since the JSON response is an array, you need to unmarshal it into a slice of structs. Make sure you also use the correct Go types. Use float64 for JSON numbers.

type Trade struct {
    TradeID      float64 `json:"id"`
    Price        string  `json:"price"`
    Qty          string  `json:"qty"`
    QuoteQty     string  `json:"quoteQty"`
    Time         float64 `json:"time"`
    IsBuyerMaker bool    `json:"isBuyerMaker"`
    IsBestMatch  bool    `json:"isBestMatch"`
}

func main() {
    symbol := "ETHEUR"
    limit := 1000
    uri := fmt.Sprintf("https://api.binance.com/api/v3/trades?symbol=%v&limit=%v", symbol, limit)
    response, err := http.Get(uri)
    if err != nil {
        fmt.Println(err)
    }
    body, err := ioutil.ReadAll(response.Body)
    if err != nil {
        fmt.Println(err)
    }
    defer response.Body.Close()
    trade := []Trade{}
    err = json.Unmarshal([]byte(body), &trade)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(trade)
}

Nacho Nieva · Accepted Answer · 2021-05-12 13:12:30Z

1

Could be helpful if we have a json sample (mocked) to know if maybe there is an exact problem with the because of the json.

I'm suspecting that because you have an ARRAY of the struct you mentioned before, you should do an unmarshal over a []Trade{} instead of a Trade{} Finally for json types, there is a kind of a rule set for when you do an unmarshal, but only when your fields ar type interface{}:

source

answered May 12, 2021 at 13:12

Nacho Nieva

1811 gold badge1 silver badge12 bronze badges

Comments

Vasuki Dileep · Accepted Answer · 2021-05-12 13:46:32Z

If you are getting json array, I presume they are array of json objects and each object contains the json as per the struct you have defined. Something like below sample.

[
  {
    "id": 1,
    "price": "111",
    "qty": "20",
    "quoteQty": "100",
    "time": 6039484,
    "isBuyerMaker": true,
    "isBestMatch": false
  },
  {
    "id": 2,
    "price": "222",
    "qty": "20",
    "quoteQty": "100",
    "time": 6039484,
    "isBuyerMaker": true,
    "isBestMatch": false
  },
  {
    "id": 3,
    "price": "333",
    "qty": "20",
    "quoteQty": "100",
    "time": 6039484,
    "isBuyerMaker": true,
    "isBestMatch": false
  }
]

In your code, you need to perform these operations.

Declare a slice of Trade struct,
Perform json unmarshall of this data to that slice
Iterate over the slice to get a handler on each json object.
Now you can form an sql query and access the json values of each json object and make a query to database.

Something like this.

// Assuming r is your default http.Request handler, get the complete request body
byteSlice, err := ioutil.ReadAll(r.Body)
    if err != nil {
        log.Println("Error in reading request body")
    }

var trades []Trade

if err = json.Unmarshal(byteSlice, &trades); err != nil {
        log.Println("Error in json unmarshal")
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }

for _, trade := range trades {
        log.Println("Trade ID: ", trade.TradeID)
        // Call a function and pass each trade object
        err := triggerSQLQuery(trade)
        if err != nil {
          log.Println(err.Error())
        }
    }


func triggerSQLQuery(trade Trade) error {
  // Access trade object and form an sql query and execute it.
}

Collectives™ on Stack Overflow

Convert JSON array API response to a struct

3 Answers 3

Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Related