PostgreSQL Index Only Scan can NOT return ctid Announcing the arrival of Valued Associate...

What computer would be fastest for Mathematica Home Edition?

Fishing simulator

Why does tar appear to skip file contents when output file is /dev/null?

Complexity of many constant time steps with occasional logarithmic steps

How to retrograde a note sequence in Finale?

When communicating altitude with a '9' in it, should it be pronounced "nine hundred" or "niner hundred"?

Problem when applying foreach loop

How are presidential pardons supposed to be used?

How to add zeros to reach same number of decimal places in tables?

Stop battery usage [Ubuntu 18]

Interesting examples of non-locally compact topological groups

How do I automatically answer y in bash script?

What items from the Roman-age tech-level could be used to deter all creatures from entering a small area?

Is above average number of years spent on PhD considered a red flag in future academia or industry positions?

Cold is to Refrigerator as warm is to?

Unable to start mainnet node docker container

I'm thinking of a number

Am I ethically obligated to go into work on an off day if the reason is sudden?

Communication vs. Technical skills ,which is more relevant for today's QA engineer positions?

Why is "Captain Marvel" translated as male in Portugal?

Estimated State payment too big --> money back; + 2018 Tax Reform

Who can trigger ship-wide alerts in Star Trek?

How can I protect witches in combat who wear limited clothing?

Is there folklore associating late breastfeeding with low intelligence and/or gullibility?



PostgreSQL Index Only Scan can NOT return ctid



Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)How do I decompose ctid into page and row numbers?Optimizing ORDER BY in a full text search queryHow to index WHERE (start_date >= '2013-12-15')How can I speed up a Postgres query containing lots of Joins with an ILIKE conditionpostgres explain plan with giant gaps between operationsSlow fulltext search due to wildly inaccurate row estimatesIndex for numeric field is not usedpostgresql 9.2 hash join issueSorting killing my postgresql queryWhy is this query with WHERE, ORDER BY and LIMIT so slow?Performance difference in accessing differrent columns in a Postgres Table





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







1















I have this table in PostgreSQL 11:



CREATE TABLE A (id bigint PRIMARY KEY, text text)


Now I want to fetch the ctid of rows meeting some condition like id = 123.



However, even using pg_hint_plan to hint PostgreSQL to use Index Only Scan, it will always give me Index Scan:



/*+ IndexOnlyScan(a) */ explain analyze select ctid from a where id = 823977776533426178;
QUERY PLAN
----------------------------------------------------------------------------------------------------------
Index Scan using a_pkey on a (cost=0.14..8.16 rows=1 width=6) (actual time=0.038..0.039 rows=1 loops=1)
Index Cond: (id = '823977776533426178'::bigint)
Planning Time: 0.122 ms
Execution Time: 0.095 ms
(4 rows)


My understanding is that ctid is the physical row id (block_no: record_offset) of each row and it must be included in any btree index, since it needs this information to fetch the row from heap file.



Then why can't an index-only scan return the ctid directly?

Is it just that PostgreSQL does not implement it that way?










share|improve this question































    1















    I have this table in PostgreSQL 11:



    CREATE TABLE A (id bigint PRIMARY KEY, text text)


    Now I want to fetch the ctid of rows meeting some condition like id = 123.



    However, even using pg_hint_plan to hint PostgreSQL to use Index Only Scan, it will always give me Index Scan:



    /*+ IndexOnlyScan(a) */ explain analyze select ctid from a where id = 823977776533426178;
    QUERY PLAN
    ----------------------------------------------------------------------------------------------------------
    Index Scan using a_pkey on a (cost=0.14..8.16 rows=1 width=6) (actual time=0.038..0.039 rows=1 loops=1)
    Index Cond: (id = '823977776533426178'::bigint)
    Planning Time: 0.122 ms
    Execution Time: 0.095 ms
    (4 rows)


    My understanding is that ctid is the physical row id (block_no: record_offset) of each row and it must be included in any btree index, since it needs this information to fetch the row from heap file.



    Then why can't an index-only scan return the ctid directly?

    Is it just that PostgreSQL does not implement it that way?










    share|improve this question



























      1












      1








      1








      I have this table in PostgreSQL 11:



      CREATE TABLE A (id bigint PRIMARY KEY, text text)


      Now I want to fetch the ctid of rows meeting some condition like id = 123.



      However, even using pg_hint_plan to hint PostgreSQL to use Index Only Scan, it will always give me Index Scan:



      /*+ IndexOnlyScan(a) */ explain analyze select ctid from a where id = 823977776533426178;
      QUERY PLAN
      ----------------------------------------------------------------------------------------------------------
      Index Scan using a_pkey on a (cost=0.14..8.16 rows=1 width=6) (actual time=0.038..0.039 rows=1 loops=1)
      Index Cond: (id = '823977776533426178'::bigint)
      Planning Time: 0.122 ms
      Execution Time: 0.095 ms
      (4 rows)


      My understanding is that ctid is the physical row id (block_no: record_offset) of each row and it must be included in any btree index, since it needs this information to fetch the row from heap file.



      Then why can't an index-only scan return the ctid directly?

      Is it just that PostgreSQL does not implement it that way?










      share|improve this question
















      I have this table in PostgreSQL 11:



      CREATE TABLE A (id bigint PRIMARY KEY, text text)


      Now I want to fetch the ctid of rows meeting some condition like id = 123.



      However, even using pg_hint_plan to hint PostgreSQL to use Index Only Scan, it will always give me Index Scan:



      /*+ IndexOnlyScan(a) */ explain analyze select ctid from a where id = 823977776533426178;
      QUERY PLAN
      ----------------------------------------------------------------------------------------------------------
      Index Scan using a_pkey on a (cost=0.14..8.16 rows=1 width=6) (actual time=0.038..0.039 rows=1 loops=1)
      Index Cond: (id = '823977776533426178'::bigint)
      Planning Time: 0.122 ms
      Execution Time: 0.095 ms
      (4 rows)


      My understanding is that ctid is the physical row id (block_no: record_offset) of each row and it must be included in any btree index, since it needs this information to fetch the row from heap file.



      Then why can't an index-only scan return the ctid directly?

      Is it just that PostgreSQL does not implement it that way?







      postgresql index postgresql-performance postgresql-11 physical-design






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited 5 mins ago









      Erwin Brandstetter

      95.9k9188305




      95.9k9188305










      asked 18 hours ago









      Qiushi BaiQiushi Bai

      133




      133






















          1 Answer
          1






          active

          oldest

          votes


















          0














          Your understanding is almost but not quite correct. True, every btree index tuple needs a ctid (or some form of block number and tuple index) to point to the heap tuple (table row). But (at least) since the introduction of "heap-only tuples" with Postgres 8.3, there may be a HOT chain to follow to arrive at the current live tuple in the snapshot - with a different ctid than the one stored in the index.



          Since it is also not possible to use the ctid (or any system column) as index expression, it is currently (and for the forseeable future) completely impossible to get a ctid from an index-only scan.



          Related:




          • Redundant data in update statements

          • How do I decompose ctid into page and row numbers?


          Quoting the Postgres Wiki on Index-only_scans / Interaction_with_HOT:




          With HOT, it became possible for an index scan to traverse a so-called
          HOT chain; it could get from the physical index tuple (which would
          probably have been created by an original INSERT, and related to an
          earlier version of the logical tuple), to the corresponding physical
          heap tuple. The heap tuple would itself contain a pointer to the next
          version of the tuple (that is, the tuple ctid), which might, in turn,
          have a pointer of its own. The index scan eventually arrives at tuple
          that is current according to the query's snapshot.







          share|improve this answer
























            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "182"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f234740%2fpostgresql-index-only-scan-can-not-return-ctid%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            0














            Your understanding is almost but not quite correct. True, every btree index tuple needs a ctid (or some form of block number and tuple index) to point to the heap tuple (table row). But (at least) since the introduction of "heap-only tuples" with Postgres 8.3, there may be a HOT chain to follow to arrive at the current live tuple in the snapshot - with a different ctid than the one stored in the index.



            Since it is also not possible to use the ctid (or any system column) as index expression, it is currently (and for the forseeable future) completely impossible to get a ctid from an index-only scan.



            Related:




            • Redundant data in update statements

            • How do I decompose ctid into page and row numbers?


            Quoting the Postgres Wiki on Index-only_scans / Interaction_with_HOT:




            With HOT, it became possible for an index scan to traverse a so-called
            HOT chain; it could get from the physical index tuple (which would
            probably have been created by an original INSERT, and related to an
            earlier version of the logical tuple), to the corresponding physical
            heap tuple. The heap tuple would itself contain a pointer to the next
            version of the tuple (that is, the tuple ctid), which might, in turn,
            have a pointer of its own. The index scan eventually arrives at tuple
            that is current according to the query's snapshot.







            share|improve this answer




























              0














              Your understanding is almost but not quite correct. True, every btree index tuple needs a ctid (or some form of block number and tuple index) to point to the heap tuple (table row). But (at least) since the introduction of "heap-only tuples" with Postgres 8.3, there may be a HOT chain to follow to arrive at the current live tuple in the snapshot - with a different ctid than the one stored in the index.



              Since it is also not possible to use the ctid (or any system column) as index expression, it is currently (and for the forseeable future) completely impossible to get a ctid from an index-only scan.



              Related:




              • Redundant data in update statements

              • How do I decompose ctid into page and row numbers?


              Quoting the Postgres Wiki on Index-only_scans / Interaction_with_HOT:




              With HOT, it became possible for an index scan to traverse a so-called
              HOT chain; it could get from the physical index tuple (which would
              probably have been created by an original INSERT, and related to an
              earlier version of the logical tuple), to the corresponding physical
              heap tuple. The heap tuple would itself contain a pointer to the next
              version of the tuple (that is, the tuple ctid), which might, in turn,
              have a pointer of its own. The index scan eventually arrives at tuple
              that is current according to the query's snapshot.







              share|improve this answer


























                0












                0








                0







                Your understanding is almost but not quite correct. True, every btree index tuple needs a ctid (or some form of block number and tuple index) to point to the heap tuple (table row). But (at least) since the introduction of "heap-only tuples" with Postgres 8.3, there may be a HOT chain to follow to arrive at the current live tuple in the snapshot - with a different ctid than the one stored in the index.



                Since it is also not possible to use the ctid (or any system column) as index expression, it is currently (and for the forseeable future) completely impossible to get a ctid from an index-only scan.



                Related:




                • Redundant data in update statements

                • How do I decompose ctid into page and row numbers?


                Quoting the Postgres Wiki on Index-only_scans / Interaction_with_HOT:




                With HOT, it became possible for an index scan to traverse a so-called
                HOT chain; it could get from the physical index tuple (which would
                probably have been created by an original INSERT, and related to an
                earlier version of the logical tuple), to the corresponding physical
                heap tuple. The heap tuple would itself contain a pointer to the next
                version of the tuple (that is, the tuple ctid), which might, in turn,
                have a pointer of its own. The index scan eventually arrives at tuple
                that is current according to the query's snapshot.







                share|improve this answer













                Your understanding is almost but not quite correct. True, every btree index tuple needs a ctid (or some form of block number and tuple index) to point to the heap tuple (table row). But (at least) since the introduction of "heap-only tuples" with Postgres 8.3, there may be a HOT chain to follow to arrive at the current live tuple in the snapshot - with a different ctid than the one stored in the index.



                Since it is also not possible to use the ctid (or any system column) as index expression, it is currently (and for the forseeable future) completely impossible to get a ctid from an index-only scan.



                Related:




                • Redundant data in update statements

                • How do I decompose ctid into page and row numbers?


                Quoting the Postgres Wiki on Index-only_scans / Interaction_with_HOT:




                With HOT, it became possible for an index scan to traverse a so-called
                HOT chain; it could get from the physical index tuple (which would
                probably have been created by an original INSERT, and related to an
                earlier version of the logical tuple), to the corresponding physical
                heap tuple. The heap tuple would itself contain a pointer to the next
                version of the tuple (that is, the tuple ctid), which might, in turn,
                have a pointer of its own. The index scan eventually arrives at tuple
                that is current according to the query's snapshot.








                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered 11 mins ago









                Erwin BrandstetterErwin Brandstetter

                95.9k9188305




                95.9k9188305






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Database Administrators Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f234740%2fpostgresql-index-only-scan-can-not-return-ctid%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Parapolítica Índice Antecedentes El escándalo Proceso judicial Consecuencias Véase...

                    How to remove border from elements in the last row?Targeting flex items on the last rowHow to vertically wrap...

                    Tecnologías entrañables Índice Antecedentes Desarrollo Tecnologías Entrañables en la...