Custom scanner for structs in Go

Question

I am writing a sort of a REST-like API in Go. The database I have is legacy, so I don't control the field names or the structure or for that matter anything. I am abstracting database access methods in a separate package called datastore. The code I have looks like this (skipping all the error handling etc.)

type Datastore struct {}

type Object struct {
  id uint
  name string
  ... zillion more fields here
}

func (Datastore) ObjectList() {
  var objects *[]Object
  db, _ := sqlx.Open("postgres", conn_info)
  rows, _ := sqlx.Queryx("SELECT * FROM object_table")
  defer rows.Close()
  for rows.Next() {
    var obj Object
    rows.Scan(&obj.id, &obj.name)
    objects = append(objects, obj)
  }
  return objects
}

The problem I am currently having is that the object table has dozens and dozens of fields. Some I care about, but some I do not. Some are named the same as Object and some are not. Eventually, I will need to support most of them, but I am working on a proof of concept first. It seems that the Scan code fails if it finds more fields in the row than are present in the arguments for Scan(). I could list the fields I scan for in the query select id, name from object_table, but 1. it makes the code extremely ugly (SQL doesn't get formatted right by gofmt) 2. it adds another place I need to edit when I want to support another field

Is there any way to implement a custom scanner interface that would take the rows object, load some data out of it into a struct and ignore the rest?

You should make it clear you are using sqlx in your question, don't make people try to guess what is going on. Furthermore that's the way the API works. You need 3 fields, scans takes 3 values, period there is no way around it. If you need another behavior use another method, like Get or Select . — mpm
– mpm, Commented Dec 22, 2016 at 6:04
I would sugest you may try an ORM like github.com/jinzhu/gorm . — Sarath Sadasivan Pillai
– Sarath Sadasivan Pillai, Commented Dec 22, 2016 at 6:59
@mpm Get() and Select() run into the same problem. What do you mean "there is no way around it"? There is always a way around it. It might be implementing my own scan, but I cannot seems to figure out how. — Mad Wombat
– Mad Wombat, Commented Dec 22, 2016 at 15:28
@Sarathsp yeah, I considered gorm, but the more complex queries seem to be awkward to implement in it — Mad Wombat
– Mad Wombat, Commented Dec 22, 2016 at 15:29
@MadWombat My point is that ultimately you are limited by sql/db API, that's how this API works. So no. Given a number of columns X you need to provide X arguments in Scan. Period . — mpm
– mpm, Commented Dec 23, 2016 at 9:18

mpm · Accepted Answer · 2016-12-24 06:13:36Z

0

You are already using sqlx so just use *DB.Unsafe() before querying :

var db *sqlx.DB

// exactly the same as the built-in
db = sqlx.Open("sqlite3", ":memory:")
var p Person
udb := db.Unsafe()
err = udb.Get(&p, "SELECT * FROM person, place LIMIT 1;")

answered Dec 24, 2016 at 6:13

mpm

20.2k7 gold badges52 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mad Wombat Over a year ago

Huh... exactly what I was looking for. This feature should be more prominent in the docs :)

Collectives™ on Stack Overflow

Custom scanner for structs in Go

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related