json is one of the most wildly used Go packages. It is simple and, what is more important, very intuitive. So what could be easier than marshalling a string with JSON and unmarshalling it to struct? If you believe (as I did) that the issue is trivial and json.Marshal does the job, read on.

What’s wrong with json.Marshal?

It’s easier to demonstrate on example. Let’s write a simple program which serializes JSON string to bytes and deserializes the bytes into matching struct:

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    FirstName string `json:"firstName"`
    LastName  string `json:"lastName"`
}

func main() {
    in := `{"firstName":"John","lastName":"Dow"}`

    bytes, err := json.Marshal(in)
    if err != nil {
        panic(err)
    }

    var p Person
    err = json.Unmarshal(bytes, &p)
    if err != nil {
        panic(err)
    }

    fmt.Printf("%+v", p)
}

When you run the program it will panic:

panic: json: cannot unmarshal string into Go value of type main.Person

goroutine 1 [running]:
panic(0x1064c0, 0xc820014240)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
main.main()
    /Users/yury/projects/go/src/exp/main.go:24 +0x194

This is unexpected. Let’s make sure that JSON string matches Person struct:

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    FirstName string `json:"firstName"`
    LastName  string `json:"lastName"`
}

func main() {
    bytes, err := json.Marshal(Person{
        FirstName: "John",
        LastName:  "Dow",
    })
    if err != nil {
        panic(err)
    }

    fmt.Println(string(bytes))
}

Which outputs:

{"firstName":"John","lastName":"Dow"}

Serialized struct is identical to our JSON string.

What’s going on?

Let’s compare serialized JSON string with result of struct serialization:

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    FirstName string `json:"firstName"`
    LastName  string `json:"lastName"`
}

func main() {
    bytes, err := json.Marshal(Person{
        FirstName: "John",
        LastName:  "Dow",
    })
    if err != nil {
        panic(err)
    }

    fmt.Println(string(bytes))

    in := `{"firstName":"John","lastName":"Dow"}`

    bytes, err = json.Marshal(in)
    if err != nil {
        panic(err)
    }

    fmt.Println(string(bytes))
}

The output reveals the truth:

{"firstName":"John","lastName":"Dow"}
"{\"firstName\":\"John\",\"lastName\":\"Dow\"}"

Can you see escaped double quotes? This is the crux of the problem. json.Marshal escapes string while serializing it.

Skipping escaping

json package includes a solution for the issue. It has RawMessage type, which Marshalls and Unmarshals without escaping. So if you need to serialize JSON string which you need to deserialize later into struct, do it with json.RawMessage:

package main

import (
    "encoding/json"
    "fmt"
)

type Person struct {
    FirstName string `json:"firstName"`
    LastName  string `json:"lastName"`
}

func main() {
    in := `{"firstName":"John","lastName":"Dow"}`

    rawIn := json.RawMessage(in)
    bytes, err := rawIn.MarshalJSON()
    if err != nil {
        panic(err)
    }

    var p Person
    err = json.Unmarshal(bytes, &p)
    if err != nil {
        panic(err)
    }

    fmt.Printf("%+v", p)
}

Output as expected:

{FirstName:John LastName:Dow}

Is there a simpler way?

Yes. json.RawMessage is just an alias to []byte. So what you really need is to convert string into []byte:

package main

import (
	"encoding/json"
	"fmt"
)

type Person struct {
	FirstName string `json:"firstName"`
	LastName  string `json:"lastName"`
}

func main() {
	in := `{"firstName":"John","lastName":"Dow"}`
	bytes := []byte(in)

	var p Person
	err := json.Unmarshal(bytes, &p)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%+v", p)
}

Output is the same:

{FirstName:John LastName:Dow}