golang 中你应该知道的noCopy策略

防止拷贝问题:Go中的不拷贝策略

To change one’s life, start immediately, do it flamboyantly, no exceptions.
— William James

1.1 sync.noCopy类型

在学习 WaitGroup 代码时:WaitGroup You Should Know in Golang | by Wesley Wei | Jul, 2024 | Programmer’s Career,我注意到了 noCopy 类型,而且看到一个很熟悉的注释:”must not be copied after first use”。

1
2
3
4
5
6
7
8
9
10
// A WaitGroup must not be copied after first use.
//
// In the terminology of the Go memory model, a call to Done
// “synchronizes before” the return of any Wait call that it unblocks.
type WaitGroup struct {
noCopy noCopy

state atomic.Uint64 // high 32 bits are counter, low 32 bits are waiter count.
sema uint32
}

通过搜索,可以发现 “must not be copied after first use” 和 noCopy 类型大都同时出现。
image.png

1
2
3
4
5
6
// Note that it must not be embedded, due to the Lock and Unlock methods.
type noCopy struct{}

// Lock is a no-op used by -copylocks checker from `go vet`.
func (*noCopy) Lock() {}
func (*noCopy) Unlock() {}

观察 noCopy Definiton In Go1.23 的定义,可以发现:

  1. noCopy 类型是空 struct
  2. noCopy 类型实现了两个方法: Lock 和 Unlock,而且都是空方法(no-op)
  3. 注释强调了 Lock 和 Unlock 在 go vet 检查时用到

noCopy类型没有任何实质的功能属性,我想只有思考、动手才知道它的具体作用,顺便弄清楚 why “must not be copied after first use”。

1.2 go vet and “locks erroneously passed by value”

当我们输入以下命令:

1
go tool vet help copylocks

可以得到相关介绍:

1
2
3
4
5
copylocks: check for locks erroneously passed by value

Inadvertently copying a value containing a lock, such as sync.Mutex or
sync.WaitGroup, may cause both copies to malfunction. Generally such
values should be referred to through a pointer.

这告诉我们当使用值传递(copying a value containing a lock)时,可能会出现意想不到的问题,具体会出现什么问题呢?举个例子:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package main

import (
"fmt"
"sync"
)

type T struct {
lock sync.Mutex
}

func (t T) Lock() {
t.lock.Lock()
}
func (t T) Unlock() {
t.lock.Unlock()
}
func main() {
var t T
t.Lock()
fmt.Println("test")
t.Unlock()
fmt.Println("finished")
}
1
2
3
4
5
6
7
8
9
10
11
12
// output
test
fatal error: sync: unlock of unlocked mutex

goroutine 1 [running]:
sync.fatal({0x4b2c9b?, 0x4a14a0?})
/usr/local/go-faketime/src/runtime/panic.go:1031 +0x18

// ❯ go vet .
# noCopy
./main.go:12:9: Lock passes lock by value: noCopy.T contains sync.Mutex
./main.go:15:9: Unlock passes lock by value: noCopy.T contains sync.Mutex

这个例子会输出错误,为什么呢?因为func (t T) Lock() 和 func (t T) Unlock() 方法使用值接收者 t。调用方法时会创建 T 的一个副本,在运行时 Lock()中的 t.lock 和 Unlock() 的t.lock 对应的不是同一个锁实例。

那么如何修复呢?copylocks 的注释里也给出了答案。使用指针接受者,这样就使用了同一个锁实例。

1
2
3
4
5
6
func (t *T) Lock() {
t.lock.Lock()
}
func (t *T) Unlock() {
t.lock.Unlock()
}

同理,当我们使用Cond、WaitGroup等时,也需要我们保证对应内容 ”must not be copied after first use“,举一个错误使用WaitGroup的例子:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
package main

import (
"fmt"
"sync"
"time"
)

func worker(id int, wg sync.WaitGroup) {
defer wg.Done()
fmt.Printf("Worker %d starting\n", id)
time.Sleep(time.Second)
fmt.Printf("Worker %d done\n", id)
}

func main() {
var wg sync.WaitGroup

for i := 1; i <= 3; i++ {
wg.Add(1)
go worker(i, wg) // passes lock by value
}

wg.Wait()

fmt.Println("All workers done!")
}
/////
Worker 3 starting
Worker 1 starting
Worker 2 starting
Worker 1 done
Worker 3 done
Worker 2 done
fatal error: all goroutines are asleep - deadlock!

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc000108040?)

// ❯ go vet .
# noCopy
./main.go:9:24: worker passes lock by value: sync.WaitGroup contains sync.noCopy
./main.go:21:16: call of worker copies lock value: sync.WaitGroup contains sync.noCopy

如何修复呢?使用同一个wg即可,读到这里不妨动手试试。 更多copylock内容参考:copylock

1.3 尝试go vet 检测

go vetnoCopy 的设计是一种避免结构体被拷贝的机制,尤其适用于那些包含同步原语(如 sync.Mutexsync.WaitGroup)的结构体。意义在于防范不当的 copylocks 发生,但这种防范不是强制的,是否应该拷贝需要开发者自行检测。例如:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
package main

import "fmt"

type noCopy struct{}

func (*noCopy) Lock() {}
func (*noCopy) Unlock() {}

type noCopyData struct {
Val int32
noCopy
}

func main() {
c1 := noCopyData{Val: 10}
c2 := c1
c2.Val = 20
fmt.Println(c1, c2)
}

上面这个例子并没用实质性的用处,程序可以正确运行但go vet 会提示“passes lock by value”问题,这里只是动手试用了 go vet 检测的机制。
但如果你需要写一些同步原语相关的代码(类似 sync.Mutexsync.WaitGroup),no Copy的机制可能对你有用。

1.4 其他noCopy策略

由上我们可知 go vet 可以检查非强制的拷贝潜在问题,那有没有强制不允许拷贝的策略呢?有的。首先看一下 strings.Builder 的源代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

// A Builder is used to efficiently build a string using [Builder.Write] methods.
// It minimizes memory copying. The zero value is ready to use.
// Do not copy a non-zero Builder.
type Builder struct {
addr *Builder // of receiver, to detect copies by value

// External users should never get direct access to this buffer, since
// the slice at some point will be converted to a string using unsafe, also
// data between len(buf) and cap(buf) might be uninitialized.
buf []byte
}

func (b *Builder) copyCheck() {
if b.addr == nil {
// This hack works around a failing of Go's escape analysis
// that was causing b to escape and be heap allocated.
// See issue 23382.
// TODO: once issue 7921 is fixed, this should be reverted to
// just "b.addr = b".
b.addr = (*Builder)(abi.NoEscape(unsafe.Pointer(b)))
} else if b.addr != b {
panic("strings: illegal use of non-zero Builder copied by value")
}
}

// Write appends the contents of p to b's buffer.
// Write always returns len(p), nil.
func (b *Builder) Write(p []byte) (int, error) {
b.copyCheck()
b.buf = append(b.buf, p...)
return len(p), nil
}

重点是:

1
b.addr = (*Builder)(abi.NoEscape(unsafe.Pointer(b)))

这行代码做了以下几件事:

  1. unsafe.Pointer(b):将 b 转换为 unsafe.Pointer,以便使用 abi.NoEscape
  2. abi.NoEscape(unsafe.Pointer(b)):通过 abi.NoEscape 函数,告诉编译器 b 这个指针不会逃逸,也就是说它可以继续在栈上分配,而不是堆上。
  3. (*Builder)(…):将 abi.NoEscape 返回的 unsafe.Pointer 再转换回 *Builder 类型,以便能够正常使用。
  4. 最后,b.addr 被设置为 b 自身的地址,这样可以防止 Builder 被拷贝(在后面有检查 b.addr != b 的逻辑)。

go1.23.0 builder.go
abi.NoEscape
针对 strings.Builder 的拷贝行为会发生panic:

1
2
3
4
5
6
7
8
9
10
11
func main() {
var a strings.Builder
a.Write([]byte("a"))
b := a
b.Write([]byte("b"))
}
// output
panic: strings: illegal use of non-zero Builder copied by value

goroutine 1 [running]:
strings.(*Builder).copyCheck(...)

Another example is sync.Cond,具体可以参考:What does “nocopy after first use” mean in golang and how | by Jing | Medium

1.5 总结

  1. 同步原语(如 sync.Mutexsync.WaitGroup)不应被拷贝,因为一旦被拷贝,它们的内部状态会被复制,从而引发并发问题。
  2. 虽然 Go 语言本身没有提供一种强制防止拷贝的机制,但noCopy 结构体提供了一种非强制的防拷贝机制,主要用于标识和go vet工具静态检测。
  3. Go语言的某些源代码在runtime时会进行 noCopy的检查并返回panic,像 strings.Buildersync.Cond

参考

Detect locks passed by value in Go | by Michał Łowicki | golangspec | Medium
What does “nocopy after first use” mean in golang and how | by Jing | Medium

更多该系列文章,参考medium链接:

https://wesley-wei.medium.com/list/you-should-know-in-golang-e9491363cd9a

English post: https://programmerscareer.com/golang-nocopy/
作者:Wesley Wei – Twitter Wesley Wei – Medium
发表日期:原文在 2024-07-13 21:26 时创作于 https://programmerscareer.com/golang-nocopy/.
版权声明:自由转载-非商用-非衍生-保持署名(创意共享3.0许可证

[置顶🔝]一些推广信息 Golang 高效处理集合 (Collection) 的库

评论

Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×