encoding/json/v2: unable to make omitempty work with pointer or Optional type with goexperiment.jsonv2

[wxiaoguang]

What did you do?

I need to Marshal some structs like

type MyType struct {
   Ptr *string `json:"ptr,omitempty"`
   Opt Optional[string] `json:"opt,omitempty"`
}

The Optional is an "optional value" implementation like other languages:

• opt.Has() to check whether it is set a value, opt.Value() gets its value.
• It has Marshal & Unmarshal related functions for JSON & YAML. Marshal check whether opt.Has():
  • if no value, then treat it as empty
  • if has value, then the "optional value" is not empty (even if the real value is empty string, the "optional value" itself is not empty)

The sample code is : #75623 (comment)

---

By using json(v1) and other YAML libraries, it works well:

• v.Ptr = nil and v.Opt = optional.None outputs {}
• v.Ptr = &str and v.Opt = optional.Some("") outputs {ptr:"",opt:""} : the pointer and optional field are not empty

What did you see happen?

But when using json(v2):

• v.Ptr = nil and v.Opt = optional.None outputs {}: it works as expected
• v.Ptr = &str and v.Opt = optional.Some("") outputs {} : the empty string (which should be treated "non-empty" for the pointer & optional value) are aggressively removed.
  • json(v2) does too much for the "empty value removal", maybe it's also better to accept user's "Marshal" func's decision?
  • Although I found a trick by using jsonv2.JoinOptions(jsonv1.OmitEmptyWithLegacySemantics(true)), overall it doesn't look good or right.

What did you expect to see?

I think json(v2) package should fix and relax the "empty value removal" behavior, and clarify the "omitempty" behavior, for example, for a pointer, only nil is empty, non-nil is not-empty. And it's better to let Marshal func to tell whether the value is empty and/or zero.

Why it is important

For API design, a non-existing field is literally different from an empty string.

• We don't want/need to output {"ptr": null} for callers if a field doesn't exist (nil-pointer, or optional.None)
• The "empty string" should be kept {ptr:"",opt:""} for callers because they are not NULL or default value in other languages.

[mvdan]

Can you provide a self-contained and small reproducer for this?

[wxiaoguang]

Can you provide a self-contained and small reproducer for this?

Two tests: TestJsonPtr and TestJsonOptional, the idea (and problem) behind them is the same:

• even if the real value of Optional is empty string, the "optional value" itself is not empty
• a non-existing field is literally different from an empty string, especially for API design and other languages

By the way, the "Optional" can also be "slice-based", for example, in Gitea: https://github.com/go-gitea/gitea/blob/main/modules/optional/option.go

---

package pkg1

import (
	json "encoding/json/v2"
	"testing"
)

// ================ TestJsonPtr  ================

type MyTypePtr struct {
	Ptr *string `json:"ptr,omitempty"`
}

func TestJsonPtr(t *testing.T) {
	s := MyTypePtr{
		Ptr: nil,
	}
	j, _ := json.Marshal(s)
	if string(j) != "{}" {
		t.Errorf("expected {}, got %s", j)
	}

	var strVal = ""
	s.Ptr = &strVal
	j, _ = json.Marshal(s)
	if string(j) != `{"ptr":""}` {
		t.Errorf(`expected {"ptr":""}, got %s`, j)
	}
}

// ================ TestJsonOptional  ================

type Optional[T any] struct {
	val T
}

func (o *Optional[T]) Has() bool {
	return o != nil
}

func (o *Optional[T]) Value() (ret T) {
	if o == nil {
		return ret
	}
	return o.val
}

func OptionalNone[T any]() *Optional[T] {
	return nil
}

func OptionalSome[T any](v T) *Optional[T] {
	return &Optional[T]{val: v}
}

func (o *Optional[T]) MarshalJSON() ([]byte, error) {
	if o == nil {
		return []byte("null"), nil
	}
	return json.Marshal(o.val)
}

type MyTypeOptional struct {
	Opt *Optional[string] `json:"opt,omitempty"`
}

func TestJsonOptional(t *testing.T) {
	s := MyTypeOptional{
		Opt: OptionalNone[string](),
	}
	j, _ := json.Marshal(s)
	if string(j) != "{}" {
		t.Errorf("expected {}, got %s", j)
	}

	s.Opt = OptionalSome("")
	j, _ = json.Marshal(s)
	if string(j) != `{"opt":""}` {
		t.Errorf(`expected {"opt":""}, got %s`, j)
	}
}