Is it good practice to use Linear Least-Squares with SMA?How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms

Multi tool use
Multi tool use

Is it good practice to use Linear Least-Squares with SMA?

Official degrees of earth’s rotation per day

Can I use USB data pins as a power source?

The meaning of 振り in 無茶振り

Violin - Can double stops be played when the strings are not next to each other?

Adventure Game (text based) in C++

What is a ^ b and (a & b) << 1?

Non-trivial topology where only open sets are closed

How are passwords stolen from companies if they only store hashes?

What exactly is this small puffer fish doing and how did it manage to accomplish such a feat?

Describing a chess game in a novel

Why one should not leave fingerprints on bulbs and plugs?

Is a party consisting of only a bard, a cleric, and a warlock functional long-term?

Examples of transfinite towers

Is there a hypothetical scenario that would make Earth uninhabitable for humans, but not for (the majority of) other animals?

Is honey really a supersaturated solution? Does heating to un-crystalize redissolve it or melt it?

Book: Young man exiled to a penal colony, helps to lead revolution

Is "upgrade" the right word to use in this context?

How do I hide Chekhov's Gun?

Fastest way to pop N items from a large dict

Relationship between sampajanna definitions in SN 47.2 and SN 47.35

Does multi-classing into Fighter give you heavy armor proficiency?

How difficult is it to simply disable/disengage the MCAS on Boeing 737 Max 8 & 9 Aircraft?

As a new Ubuntu desktop 18.04 LTS user, do I need to use ufw for a firewall or is iptables sufficient?



Is it good practice to use Linear Least-Squares with SMA?


How to correctly apply a linear trendline equationMeasuring treatment effect on top-ranked subjects selected at point in time from longitudinal dataEnsemble model performs better with worse performing consitutent models?Textbooks on linear regression with least squaresInterpreting regression and $R^2$ with small $n$Solution to force a polynomial curve to end at a specific locationLinear Regression Understanding Least SquaresCan residuals be calculated from N-point moving averages or just the regression line? Also, what is the standard way to determine regression line?Line of best fit does not look like a good fit. Why?Linear least squares algorithms













1












$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:



  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.


Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$







  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    7 hours ago
















1












$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:



  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.


Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$







  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    7 hours ago














1












1








1





$begingroup$


I have time-series (daily) data and I want to understand the general trend.



My current approach is:



  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.


Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average







share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







$endgroup$




I have time-series (daily) data and I want to understand the general trend.



My current approach is:



  1. Calculate the 7-day simple moving average.


  2. Add a line of best fit (linear least squares regression).


  3. Plot, then review metrics such as r, r^2, etc.


Question: is it good practice to draw a line-of-best fit on a moving average? I'm not very experienced but my understanding is MA and linear trend lines are both trend lines, so I'm not sure if it's OK to combine them in this way.



Raw data looks like this:



day             + count

2015-01-01 | 123

2015-01-02 | 290

2015-01-03 | 329

2015-01-04 | 276



Let me know if more detail would help- any direction on this is much appreciated.






regression time-series correlation trend moving-average




regression time-series correlation trend moving-average






share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|cite|improve this question







New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|cite|improve this question




share|cite|improve this question






New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 7 hours ago









Chef36Chef36

61




61




New contributor




Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Chef36 is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.







  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    7 hours ago













  • 1




    $begingroup$
    It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
    $endgroup$
    – whuber♦
    7 hours ago








1




1




$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
7 hours ago





$begingroup$
It depends on how you compute step (2), "a line of best fit (linear least squares regression)." If you just drop the data into a least squares black box, most likely it operates under the assumption the errors are independent, whereas in a moving average the errors are strongly interdependent (e.g., neighboring 7-day averages have six days of data in common). You need to use a procedure that accounts for this. There are robust ways to explore trends, such as various nonparametric smoothers, so maybe it would be more fruitful to investigate them rather than fixing your current approach.
$endgroup$
– whuber♦
7 hours ago











1 Answer
1






active

oldest

votes


















3












$begingroup$

Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






share|cite|improve this answer









$endgroup$












    Your Answer





    StackExchange.ifUsing("editor", function ()
    return StackExchange.using("mathjaxEditing", function ()
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    );
    );
    , "mathjax-editing");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "65"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );






    Chef36 is a new contributor. Be nice, and check out our Code of Conduct.









    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    3












    $begingroup$

    Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



    Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



    These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






    share|cite|improve this answer









    $endgroup$

















      3












      $begingroup$

      Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



      Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



      These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






      share|cite|improve this answer









      $endgroup$















        3












        3








        3





        $begingroup$

        Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



        Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



        These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.






        share|cite|improve this answer









        $endgroup$



        Of course you can do a fit on a moving average. That is your right. But the statistical diagnostics are not reliable anymore. The reason is that the IID property required in standard OLS are violated when you apply plain regression on a quantity that is highly autocorrelated.



        Your $r^2$ will be artificially high and will insinuate a false sense of statistical significance. Think about this case, instead of a moving average do a linear fit on your original data in time first. And then do it a gain, you will get 100% $r^2$.



        These models are not reliable ex-ante predictors and will have very low out-of-sample qualities.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered 7 hours ago









        Gkhan CebsGkhan Cebs

        1543




        1543




















            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.









            draft saved

            draft discarded


















            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.












            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.











            Chef36 is a new contributor. Be nice, and check out our Code of Conduct.














            Thanks for contributing an answer to Cross Validated!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397917%2fis-it-good-practice-to-use-linear-least-squares-with-sma%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            k,15,nYErsI HB0Mm X,tHVKCYtoALH4vaUfIxMoN 0Cb0H,2LN9bSX5c205iqa
            -
            cNIj eKzh9EE

            Popular posts from this blog

            What is the result of assigning to std::vector::begin()? The Next CEO of Stack OverflowWhat are the differences between a pointer variable and a reference variable in C++?What does the explicit keyword mean?Concatenating two std::vectorsHow to find out if an item is present in a std::vector?Why is “using namespace std” considered bad practice?What is the “-->” operator in C++?What is the easiest way to initialize a std::vector with hardcoded elements?What is The Rule of Three?What are the basic rules and idioms for operator overloading?Why are std::begin and std::end “not memory safe”?

            Image
            Is there a way to save my career from absolute disaster? Why didn't Khan get resurrected in the Genesis Explosion? How to make a variable always equal to the result of some calculations? What can we do to stop prior company from asking us questions? How do we know the LHC results are robust? How does the mv command work with external drives? What is ( CFMCC ) on ILS approach chart? Between two walls Should I tutor a student who I know has cheated on their homework? How do scammers retract money, while you can’t? What happens if you roll doubles 3 times then land on "Go to jail?" What is the result of assigning to std::vector<T>::begin()? How to count occurrences of text in a file? What was the first Unix version to run on a microcomputer? Return the Closest Prime Number How did people program for Consoles with multiple CPUs? What does convergence in distribution "in the Gromov–Hausdorff" sense mean? How should I support th...
            Read more

            Creating centerline of river in QGIS? The 2019 Stack Overflow Developer Survey Results Are In Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)Finding centrelines from polygons in QGIS?Splitting line into two lines with GRASS GIS?Centroid of the equator and a pointpostgis: problems creating flow direction polyline; not all needed connections are drawnhow to make decent sense from scattered river depth measurementsQGIS Interpolation on Curved Grid (River DEMs)How to create automatic parking baysShortest path creation between two linesclipping layer using query builder in QGISFinding which side of closest polyline point lies on in QGIS?Create centerline from multi-digitized roadway lines Qgis 2.18Getting bathymetric contours confined only within river banks using QGIS?

            Image
            Calculating the number of coins in money change How could chocolate be as addictive as heroin? What do you call a plan that's an alternative plan in case your initial plan fails? Python - Fishing Simulator Simulating Exploding Dice Active filter with series inductor and resistor - do these exist? How can I protect witches in combat who wear limited clothing? Am I ethically obligated to go into work on an off day if the reason is sudden? Format single node in tikzcd Didn't get enough time to take a Coding Test - what to do now? Why can't wing-mounted spoilers be used to steepen approaches? How is simplicity better than precision and clarity in prose? Is there a trick to getting spices to fix to nuts? What can I do if neighbor is blocking my solar panels intentionally? Hiding Certain Lines on Table Wolves and sheep Make it rain characters Can the prologue be the backstory of your main character? Difference between はじめる and ...
            Read more