Go 1.23 Iterators Tutorial
👋 Howdy Gophers!
Go 1.23 is fresh off the proverbial press courtesy of the Go team and with it comes the new range-over-func syntax into the language core!
This article is going to explore this new functionality and demonstrate how you can pull this into your own Go application development for fame and hopefully, some fortune to boot!
The Range-over-func Experiment
This new syntax was originally pulled into the previous Go 1.22 release as an experiment that you could play about with using the GOEXPERIMENT=rangefunc flag. It’s since matured and
has been generally accepted as a fantastic new addition to the language.
This experiment effectively allows us to define iterator functions in Go and then use the range keyword to continuously call this iterator function until there is nothing less to loop through.
What Are Iterators?
Iterators are quite a cool concept that allows us to loop through things. Whoa whoa whoa, this mind-blowing discovery might seem somewhat underwhelming.
You might be thinking “ha ha, this idiot didn’t realize we could loop through things in Go until now…”.
Hear me out though, this approach allows you to loop through things and provide elements on demand which means that they are far more memory efficient than having to pre-allocate everything you want to loop through and then looping through all that lovely pre-allocated memory before it’s actually fully needed.
Understanding the Syntax
Let’s dive into the syntax and try and build up a few different examples of iterator functions. We’ll gradually build up complexity throughout the course of this article.
Let’s start by defining a Countdown iterator that allows us to loop count down from a specific value until we hit 0.
// our iterator function takes in a value and returns a func
// that takes in another func with a signature of `func(int) bool`
func Countdown(v int) func(func(int) bool) {
// next, we return a callback func which is typically
// called yield, but names like next could also be
// applicable
return func(yield func(int) bool) {
// we then start a for loop that iterates
for i := v; i >= 0; i-- {
// once we've finished looping
if !yield(i) {
// we then return and finish our iterations
return
}
}
}
}
Calling this iterator would then look something like so:
for x := range Countdown(10) {
fmt.Println(x)
}
And when run, would produce the following output:
10
9
8
7
6
5
4
3
2
1
0
Improving on the Syntax using iter.Seq
The Go authors have thankfully included some helper definitions within the iter package which make our task of writing iterators a little simpler.
Let’s take for example the Countdown function above. We’ve got a fairly verbose function signature that returns a func(func(int) bool). This isn’t the nicest
code to reason about and could be simplified like so:
func Countdown(v int) iter.Seq[int]
The code iter.Seq[int] in the above example effectively equates to “import from the iter package the Seq[V any] definition which maps to func(yield func(V) bool)”.
Note: Seq is pronounced like ‘seek’
A More Complex Example
Repetition here is going to help this concept sink in. Let’s build a more complex example that allows us to iterate through all the elements of a slice.
We’ll define an arbitrary slice of type Employee and then a specific iterator function that will allow us to iterate through every element of said slice:
// Let's define an arbitrary struct
type Employee struct {
Name string
Salary int
}
// create a pre-defined list of employees
var Employees = []Employee{
{Name: "Elliot", Salary: 4},
{Name: "Donna", Salary: 5},
}
// let's not bring generics into this just yet...
// We can define a specific iterator for our Employee list
func EmployeeIterator(e []Employee) func(func(int, Employee) bool) {
// this will return our callback function which takes in 2 arguments
// the int which is used in the loop, and an Employee struct
return func(yield func(int, Employee) bool) {
// we then loop continuously until we've gone through
// our entire list of employees
for i := 0; i <= len(e)-1; i++ {
if !yield(i, e[i]) {
// once we've exhausted all our employees
// (hopefully not literally)
// we return
return
}
}
}
}
So this is slightly more complex, but hopefully, you’re starting to see a pattern emerging between the iterator functions.
iter.Seq2 To The Rescue
Much like our single value iterator in our Countdown example, we can use a pre-defined type definition from the iter package to simplify
how our new iterator function looks like:
func EmployeeIterator(e []Employee) iter.Seq2[int, Employee] {
return func(yield func(int, Employee) bool) {
for i := 0; i <= len(e)-1; i++ {
if !yield(i, e[i]) {
return
}
}
}
}
This is functionally equivalent to the more verbose example above this one and arguably far nicer to read. I’d implore you to default to using these helper definitions in your own code where possible.
Generics FTW
What if I told you, that we could define an iterator function that will work on any slice. If you’d said this to me 5 years ago, I’d have said “yeah, yeah, yeah… I know Go is missing the concept of generics… I still love it though”.
Today though, the biggest talking point that haters used to bring up is now long forgotten.
Let’s rub this in a little and build a generic iterator function:
// EmployeeIterator is so pre-generics. Let's change it to
// something more... generic!
func Iterate[E any](e []E) func(func(int, E) bool) {
return func(yield func(int, E) bool) {
for i := 0; i <= len(e)-1; i++ {
// to really hammer home the point about memory efficiency
// let's add a sleep here to simulate us doing some fairly
// intense computational work
time.Sleep(5 * time.Second)
if !yield(i, e[i]) {
return
}
}
}
}
And the iter.Seq2 equivalent:
func Iterate[E any](e []E) iter.Seq2[int, E] {
return func(yield func(int, E) bool) {
for i := 0; i <= len(e)-1; i++ {
time.Sleep(5 * time.Second)
if !yield(i, e[i]) {
return
}
}
}
}
We can then call this like so:
for i, employee := range Iterate(Employees) {
fmt.Printf("%d: %+v\n", i, employee)
}
And you know what the best part is? We could define an entirely different slice and pass that into our Iterate function too:
for i, val := range Iterate([]int{1, 2, 3, 4}) {
fmt.Printf("%d: %+v\n", i, val)
}
That’s generics for you, eh?
Let’s drill into the generic syntax just for those a little rusty (and possibly to reinforce my own understanding…).
The [E any] syntax immediately after our function name Iterate specifies a type parameter E which could be of type anything.
Our Iterate func then takes in a slice of type E which maps to our type parameter of type any. This broadly equates to this function
being able to take a slice that contains any data structure we’d like.
Caveats
So whilst iterators are an overall great addition to the language, they do add some complexity in the form of writing the iterator functions.
This complexity, much like that of generics, is likely to be absorbed by the library developers and general developers will likely benefit by being able to write simpler code themselves using these iterator functions.
I think it’s going to be generally quite interesting watching libraries start to adopt this syntax and make things like looping over rows in a database more straightforward.
Wrapping It Up
Ok, so in this tutorial, we’ve had a look at the new iterator syntax in Go. We’ve not quite covered how you can use it to get fame and fortune, but if you do find either of those things from this tutorial, be sure to throw some love back my way.
We started off with a simpler example, and then gradually built upon the complexity in our examples.
Additional Reading
You can check out the go 1.23 source code for the iter package here -
Go 1.23 iter package