Seen the “98% of studies were ignored!” one doing the rounds on social media. The editorial in the BMJ put it in much better terms:

“One emerging criticism of the Cass review is that it set the methodological bar too high for research to be included in its analysis and discarded too many studies on the basis of quality. In fact, the reality is different: studies in gender medicine fall woefully short in terms of methodological rigour; the methodological bar for gender medicine studies was set too low, generating research findings that are therefore hard to interpret.”

  • streetlights@lemmy.worldOP
    link
    fedilink
    English
    arrow-up
    2
    arrow-down
    3
    ·
    edit-2
    7 months ago

    101 of 103 studies were not dismissed. All systematic reviews classify their source studies based on the quality of the work. Of the 103, two were classed as high quality, 58 as moderate quality and the remaining 43 as low quality. For synthesis, only high and moderate quality studies were drawn on. That’s more than half, not 2%.

    So yes, Erin is lying.

    • Cogency@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      arrow-down
      2
      ·
      edit-2
      7 months ago

      You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies for the numbers it used to reach its conclusion. You can’t say she’s lying without that review no more than I can support Erin by reading each study that was dismissed. What I can tell you is that dismissing that many studies is not normal scientific analysis. It reeks of bias.

      • streetlights@lemmy.worldOP
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        3
        ·
        edit-2
        7 months ago

        You can’t say she’s lying until we do a systemic review of why the Cass study dismissed everything but 2 studies

        This is the lie. They didn’t dismiss all but two studies, they actually included 60. More than half of the 103 studies identified for the review.

        So yes, Erin, and now yourself, are peddling a lie.

        What I can tell you is that dismissing that many studies is not normal scientific analysis.

        It’s key part of synthesising multiple sources into a meta-analysis. Including poor quality studies dilutes the quality of the overall analysis.

        It reeks of bias.

        By design, it’s biased towards higher quality research.

        • Cogency@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          arrow-down
          2
          ·
          edit-2
          7 months ago

          Synthesis is a paragraph summary inclusion ONLY, it means they didn’t use data from the study, it is dismissal. I’m done arguing that with you.

          • streetlights@lemmy.worldOP
            link
            fedilink
            English
            arrow-up
            2
            arrow-down
            3
            ·
            7 months ago

            They have absolutely used the data from those 60 studies. You can read where they say explicitly that in the report if you cared to.

            You are utterly mistaken and firm on your conviction, these are not the qualities of skepticism.

            “Don’t seek refuge in the false security of concensus”

            • Cogency@lemmy.world
              link
              fedilink
              English
              arrow-up
              3
              arrow-down
              2
              ·
              edit-2
              7 months ago

              That’s not what synthesis means. I’ve written synthesis reports before and the data you include from those reports once you have dismissed them as inaccurate, it is an entirely selective process of whatever you want to include from them. We even have a phrase for it in law, Summarily dismissed.

              • streetlights@lemmy.worldOP
                link
                fedilink
                English
                arrow-up
                2
                arrow-down
                3
                ·
                edit-2
                7 months ago

                And of the 103 reviewed they included data from 60. It is a lie to say they “dismissed all but two.”

                Legalese is irrelevant. A systematic review of scientific literature is a different beast to “writing a few synthesis reports”.

                • Cogency@lemmy.world
                  link
                  fedilink
                  English
                  arrow-up
                  3
                  arrow-down
                  2
                  ·
                  edit-2
                  7 months ago

                  So you don’t know what you are talking about. Gotcha.

                  Synthesis reports in a scientific study when presenting data, are the parts of the report where you explain why you are dismissing data, so in this case ~98% of the data or studies. So what you just said is ~98% of the data was included in the synthesis report. that’s not inclusion of the data. That’s selective inclusion to support a conclusion. A normal scientific study can’t dismiss 98% of available data. That reveals bias.