Rendered at 12:40:00 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
layer8 12 hours ago [-]
On the general topic of dark energy and dark-energy theories, I can recommend the following two recent podcast episodes by Sean Carroll (theoretical physicist):
> A recent analysis by the Dark Energy Survey Supernova Program also finds a preference for evolving dark energy, in the same direction as DESI DR2, but with a slightly lower statistical significance (bringing the combined significance down from 4.2 sigma to only 3.2).
Is this right? It would seem that adding another finding "in the same direction" should always increase the statistical significance, even if only slightly if the new evidence is week.
Under what circumstances could additional evidence for X reduce the estimated likelihood of X?
gjm11 17 hours ago [-]
It is not true that another finding in the same direction should always increase the statistical significance.
Suppose you've got a coin; null hypothesis is that it comes up heads and tails equally often. I flip the coin 5 times and get heads every time. Probability of at least this many heads on the null hypothesis is 1/32. Now I flip the coin another 9 times and get 5 heads / 4 tails: evidence in the same direction. Between the two experiments I have 10 heads / 4 tails. Probability of at least this many heads on the null hypothesis is [(14 choose 0) + ... + (14 choose 4)] / 2^14 ~= 0.09, much bigger than 1/32.
There are also other circumstances in which even additional strong evidence for X can reduce the probability of X.
Suppose I have three hypotheses A,B,C which initially I think are all equally likely. Then two things happen that both have probability 1/2 if A is true, probability 1/4 if B is true, and probability 0 if C is true. After one of them, I should think A is true with probability 2/3. After both, I should be more confident, right?
Nope. Suppose e.g. what I'm doing is pulling balls out of a bag. Hypothesis A is "I have either a bag of red balls or a bag of blue balls, with equal probability". Hypothesis B is "I have a bag with 25% red balls, 25% blue balls, and 50% green balls". Hypothesis C is "I have a bag containing only green balls".
So I pull out a ball from the bag and it's red. That happens half the time in scenario A, 1/4 the time in scenario B, and never in scenario C, like I claimed.
I put the ball back and shake things up so I'm starting afresh, and pull out another ball. This one's blue. Again: half the time in scenario A, 1/4 the time in scenario B, never in scenario C.
But those two things can't ever both happen in scenario A, because in that scenario I have a monochromatic bag. They can both happen in scenario B. And of course neither of them can happen in scenario C.
So I got a result that (on its own) was evidence for A over the other two hypotheses, and then another result that (on its own) was evidence for A over the other two hypotheses, and the effect of both together is that I know A is false and B is true.
gjm11 32 minutes ago [-]
A couple of other remarks.
1. In the first of those cases, with the biased coin, suppose that in the second phase you flip it _thousands_ of times and it comes up with N+1 heads and N tails. Then the statistical significance of the overall bias can become arbitrarily small, and if you had a concrete alternate hypothesis along the lines of "at least 5% more likely to come up H than T" the overall evidence can be extremely strong evidence against that.
2. In the second case, with the coloured balls, note that we can readily make each bit of evidence for A > B > C much stronger than the 2 : 1 : 0 above; the bag in hypothesis B could be, say, 1% red, 1% blue, 98% green. We'd still end up with B being the only one of our viable possibilities after both observations.
3. There are real-world situations with the sort of structure we see in that second case. Suppose hypothesis A is "there is a god who not infrequently works miracles", hypothesis B is "there are no gods, but people not infrequently fabricate evidence of miracles", and hypothesis C is "nothing at all miraculous-looking ever happens". Then you see what looks like a convincing case of evidence for one god, and then what looks like a convincing case of evidence for a different one. The two gods are from different religions each of which adamantly insists that the other one is pure invention.
4. The real world is never as neat as these examples. E.g., in the gods scenario you also need to consider possibilities like "actually both those religions have some truth to them, and rivalry between their gods is why each one's followers insist that the other religion is completely imaginary" and "there is a miracle-working god, but also followers of other gods fabricate miracles" and "there are no gods and very little fabrication, but sometimes incredibly unlikely things happen just by chance" and "there are no gods but there are super-advanced aliens who enjoy messing with us" and so forth. Even in the balls-from-bags scenario -- if it happens in the real world rather than in a textbook -- you have to consider things like "I was mistaken about what ball I picked the first time" and "actually sometimes one blue ball gets into a big bag of red balls by accident" and so forth.
(A lot of the most important work in, e.g., science consists not of testing rival hypotheses but of figuring out viable hypotheses to test.)
MarkusQ 12 hours ago [-]
Thank you, that makes a lot of sense!
suuuuuuuu 13 hours ago [-]
What's referenced in the quote is a reanalysis of a single dataset (by the same team) that revised the combined significance downward.
ck2 18 hours ago [-]
to be fair, cosmology has been broken every several months lately (Hubble Tension, etc.)
I mean that's how science works, discover something new, rewrite knowledge
not like religion where you make stuff up before you know anything about anything and then force everything new to adapt to the legacy of decisions thousands of years prior
Joker_vD 18 hours ago [-]
Is it? Because it all really looks awfully close to "okay, fine, the dark energy/matter apparently also can do/be this stuff as well, just so we can write in whatever correction factors we need to make the theory fit the observations".
pdonis 18 hours ago [-]
> it all really looks awfully close to "okay, fine, the dark energy/matter apparently also can do/be this stuff as well, just so we can write in whatever correction factors we need to make the theory fit the observations".
The dark energy density is already a parameter in the model. The simplest case for such a parameter is that it's just a constant, so in the absence of evidence to the contrary, Occam's Razor led cosmologists to adopt it.
But now we have evidence that suggests that it's not a constant, so we're looking at the next simplest case, a function of time (but still constant everywhere in space at each instant of time). The article describes how the DESI data suggest that it's a slowly decreasing function of time.
What has not happened is people making up models and continuing to insist on them even after the data says otherwise. That's what the GP was saying religion does.
suuuuuuuu 13 hours ago [-]
Pedantic point, but in the time-evolving model ("model"), dark energy is not uniform across space - but the non-uniformity turns out to be negligible.
pdonis 12 hours ago [-]
> in the time-evolving model ("model"), dark energy is not uniform across space
What are you basing this on?
suuuuuuuu 12 hours ago [-]
Domain expertise. (I would ask the same of you!) Fluids always have perturbations, except in the special case of w = -1 (cosmological constant); otherwise, dropping them violates energy-momentum conservation and gauge invariance.
Sorry, not buying the argument from authority here.
> Fluids always have perturbations
Not sure I agree with this as a sweeping general claim; but in any case, my question was about what in the particular models under discussion you were basing your statement on.
> except in the special case of w = -1 (cosmological constant)
Yes, this part I agree with, a cosmological constant has to be, well, constant.
> otherwise, dropping them violates energy-momentum conservation and gauge invariance
I don't understand the argument here.
> Here are a few excerpts from DESI
Unfortunately these links don't seem to be showing me specific excerpts, just the whole paper. Can you give page/section references or equation numbers?
chowells 18 hours ago [-]
Yes, you're describing cosmology. We can't create separate test universes, so all we can do is argue over which model is the best way to explain our observations. The difference from ad-hoc reasoning is the exact same as any other science. You make predictions based on models and then try to find places where those predictions hold - or are broken. In either case, you're testing the validity of a model. And when something isn't good enough, you either adjust the model or find a new one that works better.
This isn't some slapdash random patching. Everyone involved knows that assuming that the λ in λCDM is a constant is shaky. It could be a function of time, or even location in some way. But it's simplest if it's a constant. So you start by modeling the universe as if it is. Then you determine what sorts of observations would support or contradict that, and you start making them. When you get results, you start examining what version of the model best explains those observations. And someone somewhere goes off to try find a better model than any version of λCDM. If they succeed, their model will eventually supplant it. This is how science progresses, even if the experiments are less under the control of the experimentors than they'd like. The important part that you make revisions in response to observations.
(FWIW, particle physicists are constantly frustrated that they can't find counterexamples to the Standard Model. They know it has to be incomplete, but the lack of contradictory observation leaves them no direction to try to improve it.)
bognition 18 hours ago [-]
Its fundamentally different. Science doesn't ask you to accept unknowns on faith, its provides evidence that there is something we dont yet understand and that evidence has predictive value.
If science operated like religion it would say "We dont know what dark energy is, but you have to accept it or your wrong".
Instead the argument is "Hmm... our current best understanding of the universe is lacking, when we add in these extra variables many of our predictions go from wildly inaccurate to nearly accurate. There must be something going on here we cannot observe, so lets call it "dark"."
Dark energy is "Wow, there's a factor at play here that we dont understand, without it our equations do not predict what we see, with it the equations are very accurate. Additionally we can predict multiple different things with this."
And the most important part, as soon as there is evidence that refutes the Dark Energy/Matter hypothesis scientists will quickly (over a few years, a decade at the most) walk away from it. Versus religious systems take centuries for people to shift their beliefs.
MattPalmer1086 16 hours ago [-]
No, that is not how science actually progresses, it is an idealised view of it at best. Thomas Kuhn wrote "The Structure of Scientific Revolutions" back in the 1960s. Plank's principle stated "Science progresses one funeral at a time".
Scientists are also human and will tend to defend existing ideas. In cosmology the arguments over LCDM, dark matter and dark energy have been raging for decades. Dark matter people like to say that the Bullet Cluster is incontrovertible evidence of dark matter; MOND people don't agree and point to other aspects they claim refute dark matter! It's gonna take a long time for it all to shake out.
layer8 4 hours ago [-]
MOND people have become a small minority. In addition to the galaxy cluster observations, MOND doesn’t match the CMB power spectrum, while ΛCDM does, and various very precise measurements within the solar system also contradict MOND. This was different 30-40 years ago, but since then, as evidence has strongly mounted against MOND, a lot of “shaking out” has already taken place.
MattPalmer1086 55 minutes ago [-]
Well, MOND is not a cosmological theory or relativistic - there are some different versions that attempt that. MOND itself can't be used to match the CMB spectrum.
It has always been a minority position - but it is interesting to see a different perspective. ΛCDM is not without it's own problems. Not least that when new observations contradict it, it just gets tweaked - since we still have no idea what the CDM, if it does exist, actually is. It does not have a great predictive record; it is good at explaining obervations after the fact (the CMB notwithstanding).
I am interested to know what the measurements within the solar system are that you refer to. I had thought this was far too small a scale to prove anything about MOND or CDM.
SubiculumCode 12 hours ago [-]
Not an expert, but isn't your LCDM vs MOND example assymetric? Like different kinds of data supports each model very well, and so reconciliation isn't simple?
Certainly true that the data can be viewed in many different ways. Often the same data is used by both sides!
The funeral thing isn't really a position I strongly hold; it's a pithy comment that points out science is a human endeavor and maybe not quite as objective as it is often presented.
But science as a whole is ultimately a self correcting methodology, even if it might take longer than we would want and doesnt always get it right.
https://preposterousuniverse.com/podcast/2026/06/22/358-solo...
https://preposterousuniverse.com/podcast/2026/06/29/359-solo...
Is this right? It would seem that adding another finding "in the same direction" should always increase the statistical significance, even if only slightly if the new evidence is week.
Under what circumstances could additional evidence for X reduce the estimated likelihood of X?
Suppose you've got a coin; null hypothesis is that it comes up heads and tails equally often. I flip the coin 5 times and get heads every time. Probability of at least this many heads on the null hypothesis is 1/32. Now I flip the coin another 9 times and get 5 heads / 4 tails: evidence in the same direction. Between the two experiments I have 10 heads / 4 tails. Probability of at least this many heads on the null hypothesis is [(14 choose 0) + ... + (14 choose 4)] / 2^14 ~= 0.09, much bigger than 1/32.
There are also other circumstances in which even additional strong evidence for X can reduce the probability of X.
Suppose I have three hypotheses A,B,C which initially I think are all equally likely. Then two things happen that both have probability 1/2 if A is true, probability 1/4 if B is true, and probability 0 if C is true. After one of them, I should think A is true with probability 2/3. After both, I should be more confident, right?
Nope. Suppose e.g. what I'm doing is pulling balls out of a bag. Hypothesis A is "I have either a bag of red balls or a bag of blue balls, with equal probability". Hypothesis B is "I have a bag with 25% red balls, 25% blue balls, and 50% green balls". Hypothesis C is "I have a bag containing only green balls".
So I pull out a ball from the bag and it's red. That happens half the time in scenario A, 1/4 the time in scenario B, and never in scenario C, like I claimed.
I put the ball back and shake things up so I'm starting afresh, and pull out another ball. This one's blue. Again: half the time in scenario A, 1/4 the time in scenario B, never in scenario C.
But those two things can't ever both happen in scenario A, because in that scenario I have a monochromatic bag. They can both happen in scenario B. And of course neither of them can happen in scenario C.
So I got a result that (on its own) was evidence for A over the other two hypotheses, and then another result that (on its own) was evidence for A over the other two hypotheses, and the effect of both together is that I know A is false and B is true.
1. In the first of those cases, with the biased coin, suppose that in the second phase you flip it _thousands_ of times and it comes up with N+1 heads and N tails. Then the statistical significance of the overall bias can become arbitrarily small, and if you had a concrete alternate hypothesis along the lines of "at least 5% more likely to come up H than T" the overall evidence can be extremely strong evidence against that.
2. In the second case, with the coloured balls, note that we can readily make each bit of evidence for A > B > C much stronger than the 2 : 1 : 0 above; the bag in hypothesis B could be, say, 1% red, 1% blue, 98% green. We'd still end up with B being the only one of our viable possibilities after both observations.
3. There are real-world situations with the sort of structure we see in that second case. Suppose hypothesis A is "there is a god who not infrequently works miracles", hypothesis B is "there are no gods, but people not infrequently fabricate evidence of miracles", and hypothesis C is "nothing at all miraculous-looking ever happens". Then you see what looks like a convincing case of evidence for one god, and then what looks like a convincing case of evidence for a different one. The two gods are from different religions each of which adamantly insists that the other one is pure invention.
4. The real world is never as neat as these examples. E.g., in the gods scenario you also need to consider possibilities like "actually both those religions have some truth to them, and rivalry between their gods is why each one's followers insist that the other religion is completely imaginary" and "there is a miracle-working god, but also followers of other gods fabricate miracles" and "there are no gods and very little fabrication, but sometimes incredibly unlikely things happen just by chance" and "there are no gods but there are super-advanced aliens who enjoy messing with us" and so forth. Even in the balls-from-bags scenario -- if it happens in the real world rather than in a textbook -- you have to consider things like "I was mistaken about what ball I picked the first time" and "actually sometimes one blue ball gets into a big bag of red balls by accident" and so forth.
(A lot of the most important work in, e.g., science consists not of testing rival hypotheses but of figuring out viable hypotheses to test.)
I mean that's how science works, discover something new, rewrite knowledge
not like religion where you make stuff up before you know anything about anything and then force everything new to adapt to the legacy of decisions thousands of years prior
The dark energy density is already a parameter in the model. The simplest case for such a parameter is that it's just a constant, so in the absence of evidence to the contrary, Occam's Razor led cosmologists to adopt it.
But now we have evidence that suggests that it's not a constant, so we're looking at the next simplest case, a function of time (but still constant everywhere in space at each instant of time). The article describes how the DESI data suggest that it's a slowly decreasing function of time.
What has not happened is people making up models and continuing to insist on them even after the data says otherwise. That's what the GP was saying religion does.
What are you basing this on?
Here are a few excerpts from DESI:
* https://arxiv.org/html/2404.03002v3#:~:text=Although%20a%20c...
* https://arxiv.org/html/2404.03002v3#:~:text=Since%20the%20pa...
Sorry, not buying the argument from authority here.
> Fluids always have perturbations
Not sure I agree with this as a sweeping general claim; but in any case, my question was about what in the particular models under discussion you were basing your statement on.
> except in the special case of w = -1 (cosmological constant)
Yes, this part I agree with, a cosmological constant has to be, well, constant.
> otherwise, dropping them violates energy-momentum conservation and gauge invariance
I don't understand the argument here.
> Here are a few excerpts from DESI
Unfortunately these links don't seem to be showing me specific excerpts, just the whole paper. Can you give page/section references or equation numbers?
This isn't some slapdash random patching. Everyone involved knows that assuming that the λ in λCDM is a constant is shaky. It could be a function of time, or even location in some way. But it's simplest if it's a constant. So you start by modeling the universe as if it is. Then you determine what sorts of observations would support or contradict that, and you start making them. When you get results, you start examining what version of the model best explains those observations. And someone somewhere goes off to try find a better model than any version of λCDM. If they succeed, their model will eventually supplant it. This is how science progresses, even if the experiments are less under the control of the experimentors than they'd like. The important part that you make revisions in response to observations.
(FWIW, particle physicists are constantly frustrated that they can't find counterexamples to the Standard Model. They know it has to be incomplete, but the lack of contradictory observation leaves them no direction to try to improve it.)
If science operated like religion it would say "We dont know what dark energy is, but you have to accept it or your wrong".
Instead the argument is "Hmm... our current best understanding of the universe is lacking, when we add in these extra variables many of our predictions go from wildly inaccurate to nearly accurate. There must be something going on here we cannot observe, so lets call it "dark"."
Dark energy is "Wow, there's a factor at play here that we dont understand, without it our equations do not predict what we see, with it the equations are very accurate. Additionally we can predict multiple different things with this."
And the most important part, as soon as there is evidence that refutes the Dark Energy/Matter hypothesis scientists will quickly (over a few years, a decade at the most) walk away from it. Versus religious systems take centuries for people to shift their beliefs.
Scientists are also human and will tend to defend existing ideas. In cosmology the arguments over LCDM, dark matter and dark energy have been raging for decades. Dark matter people like to say that the Bullet Cluster is incontrovertible evidence of dark matter; MOND people don't agree and point to other aspects they claim refute dark matter! It's gonna take a long time for it all to shake out.
It has always been a minority position - but it is interesting to see a different perspective. ΛCDM is not without it's own problems. Not least that when new observations contradict it, it just gets tweaked - since we still have no idea what the CDM, if it does exist, actually is. It does not have a great predictive record; it is good at explaining obervations after the fact (the CMB notwithstanding).
I am interested to know what the measurements within the solar system are that you refer to. I had thought this was far too small a scale to prove anything about MOND or CDM.
Moreover, your objection that science progresses one funeral at a time is exaggerated, even if it is real, it's not necessarily all that strong: https://pmc.ncbi.nlm.nih.gov/articles/PMC6814193/
The funeral thing isn't really a position I strongly hold; it's a pithy comment that points out science is a human endeavor and maybe not quite as objective as it is often presented.
But science as a whole is ultimately a self correcting methodology, even if it might take longer than we would want and doesnt always get it right.