Is it good practice to use Linear Least-Squares with SMA?How to correctly apply a linear trendline...

What exactly is this small puffer fish doing and how did it manage to accomplish such a feat?

How do I change two letters closest to a string and one letter immediately after a string using Notepad++?

What is "focus distance lower/upper" and how is it different from depth of field?

Is a party consisting of only a bard, a cleric, and a warlock functional long-term?

I got the following comment from a reputed math journal. What does it mean?

World War I as a war of liberals against authoritarians?

Violin - Can double stops be played when the strings are not next to each other?

Are ETF trackers fundamentally better than individual stocks?

Is "upgrade" the right word to use in this context?

Simplify an interface for flexibly applying rules to periods of time

Professor being mistaken for a grad student

Employee lack of ownership

This word with a lot of past tenses

Why do passenger jet manufacturers design their planes with stall prevention systems?

What is the significance behind "40 days" that often appears in the Bible?

Instead of a Universal Basic Income program, why not implement a "Universal Basic Needs" program?

Why Choose Less Effective Armour Types?

Why does overlay work only on the first tcolorbox?

Are all passive ability checks floors for active ability checks?

Python if-else code style for reduced code for rounding floats

Is there a hypothetical scenario that would make Earth uninhabitable for humans, but not for (the majority of) other animals?

Happy pi day, everyone!

combinatorics floor summation

Recruiter wants very extensive technical details about all of my previous work



Is it good practice to use Linear Least-Squares with SMA?


How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms













1












$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$








  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago


















1












$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$








  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago
















1












1








1





$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$




I have time-series (daily) data and I want to understand the general trend.



My current approach is:




  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.



Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average




regression time-series correlation trend moving-average






share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|cite|improve this question




share|cite|improve this question






New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 2 hours ago









Chef36Chef36

61




61




New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.








  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago
















  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    2 hours ago










1




1




$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago






$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
2 hours ago












1 Answer
1






active

oldest

votes


















2












$begingroup$

Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






share|cite|improve this answer









$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "65"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });






    Chef36 is a new contributor. Be nice, and check out our Code of Conduct.










    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    2












    $begingroup$

    Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



    Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



    These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






    share|cite|improve this answer









    $endgroup$


















      2












      $begingroup$

      Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



      Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



      These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






      share|cite|improve this answer









      $endgroup$
















        2












        2








        2





        $begingroup$

        Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



        Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



        These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






        share|cite|improve this answer









        $endgroup$



        Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



        Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



        These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered 2 hours ago









        Gkhan CebsGkhan Cebs

        1443




        1443






















            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.










            draft saved

            draft discarded


















            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.













            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.












            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.
















            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            -

            Popular posts from this blog

            ORA-01691 (unable to extend lob segment) even though my tablespace has AUTOEXTEND onORA-01692: unable to...

            Image
            Do I need to convey a moral for each of my blog post? Does convergence of polynomials imply that of its coefficients? Are hand made posters acceptable in Academia? How to read string as hex number in bash? Have the tides ever turned twice on any open problem? How do researchers send unsolicited emails asking for feedback on their works? How to determine the greatest d orbital splitting? When did hardware antialiasing start being available? How to test the sharpness of a knife? Asserting that Atheism and Theism are both faith based positions Print a physical multiplication table
            Read more

            Always On Availability groups resolving state after failover - Remote harden of transaction...

            Image
            Should corporate security training be tailored based on a users' job role? Does human life have innate value over that of other animals? Limit involving inverse functions Father gets chickenpox, but doesn't infect his two children. How is this possible? Is it possible to re-order a List of continous entries? Who, if anyone, was the first astronaut to return to earth in a different vessel? The totem pole can be grouped into Identical projects by students at two different colleges: still plagiarism? Sci fi book, man buys a beat up spaceship and intervenes in a civil war on a planet and eventually becomes a space cop How to not
            Read more

            Circunscripción electoral de Guipúzcoa Referencias Menú de navegaciónLas claves del sistema electoral en...

            Image
            Circunscripciones electorales del Senado de EspañaCircunscripciones electorales del Congreso de los DiputadosCircunscripciones electorales del Parlamento Vasco españolasCongreso de los DiputadosSenadobajaaltaPaís Vascoelecciones autonómicasprovincia homónima Localización de la circunscripción electoral de Guipúzcoa. Guipúzcoa es una de las 52 circunscripciones electorales españolas utilizadas como distritos electorales para el Congreso de los Diputados y el Senado, que son las cámaras baja y alta del Parlamento Español. [ 1 ] ​ Le corresponden 6 diputados y 4 senadores. También es una de las 3 circunscripciones electorales del País Vasco para las elecciones autonómicas, en que elige 25 parlamentarios. Se corresponde con la provincia homónima. [ 2 ] ​ Referencias ↑ Las claves del sistema electoral en España ↑ Resultados de las elecciones autonómicas de 2012 en Guipúzcoa This page is only for reference, If you need detailed information, please check here
            Read more