It does not seem to cover the Neural Accelerators, Apple's equivalent of the Tensor Cores. They only got released on M5 platform. This is probably the most important part to cover.
This is obvious Claude slop writing, the author would be advised to use vale [1] with samples of their own writing as a guide.
> Performance begins with the roofline. On the M1 the engine holds about 12 fp16 TFLOP/s of compute against
a DRAM-bandwidth ceiling. The roofline has a ridge point near 141 FLOP per byte, a 2 MB working-set
threshold, a 0.23 ms floor under any single dispatch, and efficiency near 0.37 picojoules per FLOP at the
compute optimum. On a 256-channel 3x3 convolution it runs about 3.8 times faster than the same chip’s
GPU and 9 times more energy-efficient. The roofline pairs the engine’s throughput ceilings with its measured
power.
> Reaching the engine is not the same as running an arbitrary graph on it. The operations the engine executes
are distinct from the ones a capability bit only advertises. A feature attested in the hardware tables or
accepted by the compiler frontend counts only once a compile-and-run confirms it, and several advertised
operations, three-dimensional convolution among them, never lower to the engine at all. Weight compression
on the direct path cuts bandwidth, not only stored size. On the unentitled engine, int4 lookup-table weights
run about 2.37 times faster than fp16, and structured sparsity 1.55 to 1.64 times faster at 0.43 times the
bytes.
1. It uses non-idiomatic terminology in several places.
2. It repeats the same finding over and over (141 flops per byte, for example), without going deeper.
3. I stopped reading about a quarter of the way through because it felt like it was never going to stop teasing me about what it was going to tell me and actually tell me it.
4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited).
For example, I understand some of what it is saying because I used some similar techniques to benchmark things in the past (running at multiple scales to estimate overhead + marginal gains with a linear regression), but I wouldn't expect anyone who hasn't personally done that to follow the prose.
> 4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited
I've had this complaint well before LLMs were used. People writing about topics they have a lot of knowledge in the subject tend to make the assumption only other subject knowledgeable readers will read it. Or that it never edited by a real editor that would enforce rules like spelling out acronyms on first use. Or forcing additional information when too many details have been left out on the assumption it would already be known.
There's plenty of this type of writing to have trained the bots that way
You obviously haven't read it, because it is clunky garbage.
> 19.4 Pacing compiles after a failure
> A failed compile is not free of side effects on the shared compile service. A compile that fails restarts the
service, which takes a few seconds to come back, and failures that keep arriving faster than the service can
restart between them keep it from making progress, so unrelated compiles slow down until the failures stop.
The effect is a function of how fast failures arrive, not how many occur: failures spaced out past the restart
interval cause no degradation at all. On detecting a failed compile, wait at least one restart interval, roughly
15 seconds, before the next compile, so a burst of failures cannot accumulate. No hard failure-count cap is
needed.
The whole document is less nutritious than a wonderbread miracle whip sandwich.
It does not seem to cover the Neural Accelerators, Apple's equivalent of the Tensor Cores. They only got released on M5 platform. This is probably the most important part to cover.
Those are part of the GPU not the Neural Engine.
This Neural Engine seems useless for LLMs. Trapped in the wrong architecture
Apple is releasing CoreAI which is supposed to be optimized for LLMs and the transformer architecture.
This scans very much as AI-written.
This is obvious Claude slop writing, the author would be advised to use vale [1] with samples of their own writing as a guide.
> Performance begins with the roofline. On the M1 the engine holds about 12 fp16 TFLOP/s of compute against a DRAM-bandwidth ceiling. The roofline has a ridge point near 141 FLOP per byte, a 2 MB working-set threshold, a 0.23 ms floor under any single dispatch, and efficiency near 0.37 picojoules per FLOP at the compute optimum. On a 256-channel 3x3 convolution it runs about 3.8 times faster than the same chip’s GPU and 9 times more energy-efficient. The roofline pairs the engine’s throughput ceilings with its measured power.
> Reaching the engine is not the same as running an arbitrary graph on it. The operations the engine executes are distinct from the ones a capability bit only advertises. A feature attested in the hardware tables or accepted by the compiler frontend counts only once a compile-and-run confirms it, and several advertised operations, three-dimensional convolution among them, never lower to the engine at all. Weight compression on the direct path cuts bandwidth, not only stored size. On the unentitled engine, int4 lookup-table weights run about 2.37 times faster than fp16, and structured sparsity 1.55 to 1.64 times faster at 0.43 times the bytes.
https://vale.sh/
Please no. The author would be advised to write their own original thoughts.
It was a joke, nothing could save this "paper". I don't think the author wrote anything. They pointed claude at a directory and said "write a paper"
why?
1. It uses non-idiomatic terminology in several places.
2. It repeats the same finding over and over (141 flops per byte, for example), without going deeper.
3. I stopped reading about a quarter of the way through because it felt like it was never going to stop teasing me about what it was going to tell me and actually tell me it.
4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited).
For example, I understand some of what it is saying because I used some similar techniques to benchmark things in the past (running at multiple scales to estimate overhead + marginal gains with a linear regression), but I wouldn't expect anyone who hasn't personally done that to follow the prose.
> 4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited
I've had this complaint well before LLMs were used. People writing about topics they have a lot of knowledge in the subject tend to make the assumption only other subject knowledgeable readers will read it. Or that it never edited by a real editor that would enforce rules like spelling out acronyms on first use. Or forcing additional information when too many details have been left out on the assumption it would already be known.
There's plenty of this type of writing to have trained the bots that way
Cmd-F for "AI" has 1000+ hits!
The burden of proof should be with the beholder. Must be so easy to scream AI when you don’t want to read an article.
You obviously haven't read it, because it is clunky garbage.
> 19.4 Pacing compiles after a failure
> A failed compile is not free of side effects on the shared compile service. A compile that fails restarts the service, which takes a few seconds to come back, and failures that keep arriving faster than the service can restart between them keep it from making progress, so unrelated compiles slow down until the failures stop. The effect is a function of how fast failures arrive, not how many occur: failures spaced out past the restart interval cause no degradation at all. On detecting a failed compile, wait at least one restart interval, roughly 15 seconds, before the next compile, so a burst of failures cannot accumulate. No hard failure-count cap is needed.
The whole document is less nutritious than a wonderbread miracle whip sandwich.
Is there a non-slop version of this information available?
I am reading up on GPU / ML micro architecture and am looking for some good sources.
I skimmed through it, what makes you think it is slop?