I need a generator and rust doesn't have generators... sigh

"extra unstable feature" on nightly 🤪

Generators should be pretty stable since I believe they share a lot of code with async in the compiler

@kobenz Oh right it literally says that :P

Surprises me though, since it's literally the same* compiler logic as for async

It's a method with a for loop. 😂

@blindXfish state machines with nameless states!

now you're really making me want generators!

@blindXfish that is what I wanted the generator for though

a method that does a for loop and I wanted to take the for loop out because it would've been cleaner. but alas

@jestdotty There are crates which give you generators on stable (by using async under the hood), just search coroutine on crates.io

`genawaiter` for example

@12bitfloat well this will have to go into wasm

actually it works in wasm

urghh do I wanna thouughh

@jestdotty You do you :P

As far as I can tell it's a pretty small dependency. Sometimes I really think you are allergic to abstractions that make your live easier

Ah rust still catching up to C++ ;P

They missed the chance to call the nightly version the UB version... Have fun making whatever you're making

@BordedDev C++ got async and generators *after* Rust

...well maybe not generators :P

Do you need sync generators or can you do async?

I have a very simple async generator.

https://eu2.contabostorage.com/b5fc... 2025-10-13 092129.png

I'm actually not sure whether this is cancellation safe, now thinking of it I'm pretty sure it yields once between generating an item and returning it, so if it got dropped during that time then one item would be lost. But cancellation safety is a bitch so I think the best approach is to only ever cancel futures that represent a unit of work with very well known cancellation safe boundaries, and assume that a general future isn't cancellation safe.

Are generators comparable to C# yield?

@12bitfloat You mean back in 2011, before rust's v1 release? Guess you should double check ;P

absolute bullshit who puts fucking spaces in a URL god damn it I can't believe the S3 protocol has been standardized.

https://eu2.contabostorage.com/b5fc...

By the way, you can hack futures to use them as a sync generator via a skeletal executor since the state machine and yielding mechanism is the same. The only minor problem is if you do that you're not allowed to await any future inside the generator besides the handle's "yield" operation since you probably wouldn't have a Waker, and there's no way to impose this restriction on usercode through the compiler. I don't think that's a showstoppoer but to each their own.

Not really sure what generators are and I assume that is what C# "yield" keyword is for, but…

can‘t you just use lambdas/closures which are called by the "generator" and "yield" the values this way?

Is it disregarded because it‘s hard to make memory safe?

@Lensflare A generator can have multiple yield points, and yield to other generators, even recursively, so the value of local variables, the instruction pointer, and even the generator call stack all have to be stored in a big compiler-generated state machine.

Just a lambda isn't enough for this, but any version of async-await syntax sugar should be able to represent it with various tradeoffs. Rust's async support in particular is a very good fit with almost no performance downsides and a low cost to ergonomics.

@lorentz I don’t quite get it. The receiver of the values that it gets from the generator doesn’t care about the inner state of the generator, does it?
All it cares about is getting the values.
So why is a lambda not enough?

func integerGenerator(lambda: (Int) -> ()) {
while(true) {
lambda(rand())
}
}

@Lensflare oh you mean like that! I gave the wrong reasons then, I thought you meant to turn the generator body into a lambda that's repeatedly invoked to generate successive elements.

What you're proposing wouldn't work because the caller can store the generator, move it around in memory, return it, or dispose of it without continuing.

@Lensflare for example, a textbook use case is to call two generators of sorted elements and then merge-sort their output as it's received without retaining the already sorted head of the sequences.

@lorentz hm… then you could put that function into a struct/class and use that as the owner of the generator?

@Lensflare well yeah then that struct/class would be the state machine. The point of a generator is that the compiler takes care of converting your function into a state machine.

@lorentz just made a little research on that…

So apparently generators only start to make sense when you also have some mechanism for suspension and continuation of execution.

So practically coroutines and/or async/await.

And when you have that, it‘s just a small step and very easy to mimic generators with closures.

It is very straightforward in Swift, so I‘d assume that it’s also doable in Rust.

Basically "I need generators but Rust doesn’t have it" is a somewhat silly attitude.
Most likely you don’t actually need real generators and you can make it work with async and closures.

@Lensflare Well, in an async context you don't ever need a generator because you can do the same thing with a parallel task and an unbuffered SPSC channel that prevents the sender from continuing until the current message has been received. That's what the snippet on the picture I linked does.

Generators or coroutines are a bit different from async in that they can be called from any function since their public interface can't express waiting for an external event. Rust's Future trait has this capability in the form of the Waker, but you are free to ignore or stub this feature as long as you ensure that you don't ever yield to another async function that uses it, so you can technically accept an async function and then only use it as a compiler-generated state machine.

@BordedDev Not sure what you're referring to. coroutines were only added in C++20

the inner state matters with mine

actually I'm an idiot

I can just use an anonymous function to handle each record

@jestdotty that’s what I was referring to with lambdas/closures

though it's not as pretty or readable

@Lensflare To be fair, even if she *did* need actual generators, there are good solutions in Rust

@12bitfloat Async was add in c++11

@BordedDev Oh yeah you're right. Though I wonder what the difference between std::async and std::coroutine is

I think only the latter is "true" stack-machine based async comparable to Rust

@12bitfloat C++ coroutines are stackless, was actually planning on using them for isspam, but I was fighting io_uring instead, and now I don't have time until I'm familiar with spring

@BordedDev Shucks!

I was planning on adding some io_uring too. I think just the file reading (and waiting...) is like half the runtime :P

@12bitfloat Probably more, but good luck with it. I got good performance, but it got "killed" by liburing's use of atomics, and I haven't gotten round to stripping it out yet

@BordedDev Haven't tried yet but my idea was to have two working buffers per thread and then submit a read for the next file while processing the current one

Not sure if you did something similar

@12bitfloat ;P