Concurrent map and slice types in Go
Go is a language well known for it’s concurrency primitives.
While Go has some really nice features making it so easy for developers to create concurrent applications, not all of the types in Go are safe for concurrent use.
For instance two of the most commonly used types in Go - slice and map - cannot be used safely from multiple goroutines without the risk of having a race condition.
On other hand using the basic synchronization primitives which Go provides to us we can create our own thread safe types.
In this post we will see how to create our own slice and map types which can be safely shared between multiple goroutines.
In order to do that we need to ensure that access to the shared data is protected by a mutex.
What a mutex does is basically to acquire a lock when it needs to access our concurrent slice or map. While holding the lock a goroutine can read and/or write to the shared data protected by the mutex.
A mutex lock can be acquired by a single goroutine at a time, and if other goroutine needs access to the same shared data it waits until the lock has been released by the goroutine holding the lock.
Forgetting to release the lock or improper use of it often leads to deadlocks. That is why we need to make sure we always release the mutex lock once we are done with it.
Let’s start now by creating our concurrent types. We will start with creating the type for concurrent slice and then make our way to the concurrent map as well.
// Slice type that can be safely shared between goroutines
type ConcurrentSlice struct {
sync.RWMutex
items []interface{}
}
// Concurrent slice item
type ConcurrentSliceItem struct {
Index int
Value interface{}
}
In order to protect access to the data in our slice we have used a sync.RWMutex using composition.
The internal slice ConcurrentSlice.items
that we use for
storing slice items is of type
interface{} which
allows us to use this concurrent slice to store any data into it.
We have also created a ConcurrentSliceItem
type which we will
use later on when we need to return an item from our concurrent slice
while iterating over it using the
builtin keyword range
.
Now, let’s implement a method which allows us to add new items to the slice.
// Appends an item to the concurrent slice
func (cs *ConcurrentSlice) Append(item interface{}) {
cs.Lock()
defer cs.Unlock()
cs.items = append(cs.items, item)
}
What this method does is to acquire the mutex lock and append the item we have passed to it and after that it releases the lock. Simple enough.
Now let’s implement a method that iterates over our slice items.
// Iterates over the items in the concurrent slice
// Each item is sent over a channel, so that
// we can iterate over the slice using the builin range keyword
func (cs *ConcurrentSlice) Iter() <-chan ConcurrentSliceItem {
c := make(chan ConcurrentSliceItem)
f := func() {
cs.Lock()
defer cs.Unlock()
for index, value := range cs.items {
c <- ConcurrentSliceItem{index, value}
}
close(c)
}
go f()
return c
}
The ConcurrentSlice.Iter()
method returns a
channel over which
ConcurrentSliceItem
items are being sent to.
This allows us to use the builtin range
keyword and iterate
over the items in our concurrent slice.
It is also worth noting that since our slice items are stored as
interface{}
in order to retrieve the underlying values we
need to use type assertions.
Let’s create now the concurrent map type.
// Map type that can be safely shared between
// goroutines that require read/write access to a map
type ConcurrentMap struct {
sync.RWMutex
items map[string]interface{}
}
// Concurrent map item
type ConcurrentMapItem struct {
Key string
Value interface{}
}
Similar to the ConcurrentSlice
type above we are using
composition to embed the
sync.RWMutex in our
ConcurrentMap
type.
When we retrieve items from the map we will be returning a
ConcurrentMapItem
type which contains the map item key and value.
Let’s implement a method for setting a new key in our concurrent map.
// Sets a key in a concurrent map
func (cm *ConcurrentMap) Set(key string, value interface{}) {
cm.Lock()
defer cm.Unlock()
cm.items[key] = value
}
And now we implement the method for retrieving a key from the map:
// Gets a key from a concurrent map
func (cm *ConcurrentMap) Get(key string) (interface{}, bool) {
cm.Lock()
defer cm.Unlock()
value, ok := cm.items[key]
return value, ok
}
The result of ConcurrentMap.Get()
method would return the
value of requested key and a boolean indicating whether the
key is present or not in the map.
And finally let’s implement a method to iterate over all items in the concurrent map.
// Iterates over the items in a concurrent map
// Each item is sent over a channel, so that
// we can iterate over the map using the builtin range keyword
func (cm *ConcurrentMap) Iter() <-chan ConcurrentMapItem {
c := make(chan ConcurrentMapItem)
f := func() {
cm.Lock()
defer cm.Unlock()
for k, v := range cm.items {
c <- ConcurrentMapItem{k, v}
}
close(c)
}
go f()
return c
}
Above method returns a channel over which ConcurrentMapItem
items are being sent to and we can use the builtin range
keyword to
iterate over all items in the map.
Similar to the concurrent slices and the fact that we have used
interface{}
to store the map values we need to use
type assertions
in order to retrieve the underlying value.
You can also find the code used in this post in the gru.utils package in Github.