first spotted in publishing-bot run:
https://github.com/kubernetes/kubernetes/issues/56876#issuecomment-2070999174

followed steps in [flaky-tests.md](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/flaky-tests.md) to recreate the problem.
- `go test ./staging/src/k8s.io/client-go/scale -race -c`
- `stress ./scale.test -test.run TestRoundTrip`

left it running overnight and can see the issue:
https://paste.mozilla.org/SQ7rRkd0/raw

couple of examples from the stress run:
```
--- FAIL: TestRoundTrip (0.07s)
    roundtrip.go:135: starting group ""
    roundtrip.go:148: finished group ""
    roundtrip.go:135: starting group "apps"
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:148: finished group "apps"
    roundtrip.go:135: starting group "autoscaling"
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:148: finished group "autoscaling"
    roundtrip.go:135: starting group "extensions"
    roundtrip.go:254: 	round tripping to extensions/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:360: Scale: diff:   &scheme.Scale{
          	TypeMeta: {},
          	ObjectMeta: v1.ObjectMeta{
          		... // 12 identical fields
          		OwnerReferences: nil,
          		Finalizers:      nil,
        - 		ManagedFields:   []v1.ManagedFieldsEntry{},
        + 		ManagedFields:   nil,
          	},
          	Spec: {Replicas: 1622107388},
          	Status: scheme.ScaleStatus{
          		Replica
…

/var/folders/q1/kqwmr8293v7_1wrzjtyvzjs00000gr/T/go-stress-20240422T195237-1690711203
--- FAIL: TestRoundTrip (0.06s)
    roundtrip.go:135: starting group ""
    roundtrip.go:148: finished group ""
    roundtrip.go:135: starting group "apps"
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:148: finished group "apps"
    roundtrip.go:135: starting group "autoscaling"
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:148: finished group "autoscaling"
    roundtrip.go:135: starting group "extensions"
    roundtrip.go:254: 	round tripping to extensions/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:360: Scale: diff:   &scheme.Scale{
          	TypeMeta: {},
          	ObjectMeta: v1.ObjectMeta{
          		... // 12 identical fields
          		OwnerReferences: {{APIVersion: "Xȩ", Kind: `\FZ`, Name: "ʮʩȼŚ尺籴ŪZɨ蓻", UID: "}mǠʉjƱ/`Ï8Èŝ0", ...}},
          		Finalizers:      {"ǒ賾赣"},
        - 		ManagedFields:   []v1.ManagedFieldsEntry{},
        + 		ManagedFields:   nil,

…
```
TestRoundTrip is flaky
This is a GitHub Issue
repo:kubernetes
owner:kubernetes
Title : TestRoundTrip is flaky
Issue date:

--- start body ---
first spotted in publishing-bot run:
https://github.com/kubernetes/kubernetes/issues/56876#issuecomment-2070999174

followed steps in [flaky-tests.md](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/flaky-tests.md) to recreate the problem.
- `go test ./staging/src/k8s.io/client-go/scale -race -c`
- `stress ./scale.test -test.run TestRoundTrip`

left it running overnight and can see the issue:
https://paste.mozilla.org/SQ7rRkd0/raw

couple of examples from the stress run:
```
--- FAIL: TestRoundTrip (0.07s)
    roundtrip.go:135: starting group ""
    roundtrip.go:148: finished group ""
    roundtrip.go:135: starting group "apps"
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:148: finished group "apps"
    roundtrip.go:135: starting group "autoscaling"
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:148: finished group "autoscaling"
    roundtrip.go:135: starting group "extensions"
    roundtrip.go:254: 	round tripping to extensions/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:360: Scale: diff:   &scheme.Scale{
          	TypeMeta: {},
          	ObjectMeta: v1.ObjectMeta{
          		... // 12 identical fields
          		OwnerReferences: nil,
          		Finalizers:      nil,
        - 		ManagedFields:   []v1.ManagedFieldsEntry{},
        + 		ManagedFields:   nil,
          	},
          	Spec: {Replicas: 1622107388},
          	Status: scheme.ScaleStatus{
          		Replica
…

/var/folders/q1/kqwmr8293v7_1wrzjtyvzjs00000gr/T/go-stress-20240422T195237-1690711203
--- FAIL: TestRoundTrip (0.06s)
    roundtrip.go:135: starting group ""
    roundtrip.go:148: finished group ""
    roundtrip.go:135: starting group "apps"
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta2, Kind=Scale v1beta2.Scale
    roundtrip.go:254: 	round tripping to apps/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:148: finished group "apps"
    roundtrip.go:135: starting group "autoscaling"
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:254: 	round tripping to autoscaling/v1, Kind=Scale v1.Scale
    roundtrip.go:148: finished group "autoscaling"
    roundtrip.go:135: starting group "extensions"
    roundtrip.go:254: 	round tripping to extensions/v1beta1, Kind=Scale v1beta1.Scale
    roundtrip.go:360: Scale: diff:   &scheme.Scale{
          	TypeMeta: {},
          	ObjectMeta: v1.ObjectMeta{
          		... // 12 identical fields
          		OwnerReferences: {{APIVersion: "Xȩ", Kind: `\FZ`, Name: "ʮʩȼŚ尺籴ŪZɨ蓻", UID: "}mǠʉjƱ/`Ï8Èŝ0", ...}},
          		Finalizers:      {"ǒ賾赣"},
        - 		ManagedFields:   []v1.ManagedFieldsEntry{},
        + 		ManagedFields:   nil,

…
```
--- end body ---
