trend analysis in research pdf

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Trend analysis for national surveys: Application to all variables from the Canadian Health Measures Survey cycle 1 to 4

Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft

Current address: Canadian Agency for Drugs and Technologies in Health, Ottawa, Ontario, Canada

Affiliation Centre de recherche du centre hospitalier de l’Université de Montréal (CRCHUM), Université de Montréal, Montréal, Québec, Canada

Roles Writing – review & editing

Affiliation Département d'informatique, Université du Québec à Montréal, Montréal, Québec, Canada

Affiliation Taipei Hospital, Ministry of Health and Welfare, New Taipei city, Taiwan

* E-mail: [email protected]

Affiliations Department of Chest Medicine, Taipei Veterans General Hospital, Taipei, Taiwan, Faculty of Medicine, School of Medicine, National Yang-Ming University, Taipei, Taiwan

• Yi-Sheng Chao, 
• Chao-Jung Wu, 
• Hsing-Chien Wu, 
• Wei-Chih Chen

• Published: August 9, 2018
• https://doi.org/10.1371/journal.pone.0200127
• Reader Comments

Trend analysis summarizes patterns over time in the data to show the direction of change and can be used to investigate uncertainties in different time points and associations with other factors. However, this approach is not widely applied to national surveys and only selected outcomes are investigated. This study demonstrates a research framework to conduct trend analysis for all variables in a national survey, the Canadian Health Measures Survey (CHMS).

Data and methods

The CHMS cycle 1 to 4 was implemented between 2007 and 2015. The characteristics of all variables were screened and associated to the weight variables. Missing values were identified and cleaned according to the User Guide. The characteristics of all variables were extracted and used to guide data cleaning. Trend analysis examined the statistical significance of candidate predictors: the cycles, age, sex, education, household income and body mass index (BMI). R (v3.2) and RStudio (v0.98.113) were used to develop the framework.

There were 26557 variables in 79 data files from four cycles. There were 1055 variables significantly associated with the CHMS cycles and 2154 associated with the BMI after controlling for other predictors. The trend of blood pressure was similar to those published.

Trend analysis for all variables in the CHMS is feasible and is a systematic approach to understand the data. Because of trend analysis, we have detected data errors and identified several environmental biomarkers with extreme rates of change across cycles. The impact of these biomarkers has not been well studied by Statistics Canada or others. This framework can be extended to other surveys, especially the Canadian Community Health Survey.

Citation: Chao Y-S, Wu C-J, Wu H-C, Chen W-C (2018) Trend analysis for national surveys: Application to all variables from the Canadian Health Measures Survey cycle 1 to 4. PLoS ONE 13(8): e0200127. https://doi.org/10.1371/journal.pone.0200127

Editor: Antonio Palazón-Bru, Universidad Miguel Hernandez de Elche, SPAIN

Received: June 26, 2017; Accepted: May 16, 2018; Published: August 9, 2018

Copyright: © 2018 Chao et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data underlying this study are CHMS data belonging to Statistics Canada. The authors did not have any special access to the CHMS data. It is against the Statistics Act of Canada to release data that are de-identified. The CHMS data is available through the Research Data Centres program administered by Statistics Canada (see this link for eligibility and detailed process to request access: https://www.statcan.gc.ca/eng/rdc/index ). Data access needs to be approved by Statistics Canada, and any analysis output needs to be vetted by Statistics Canada before being released.

Funding: Funded by Fonds de Recherche du Québec - Santé (CA) Postdoctoral fellowship to Yi-Sheng Chao. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Trend analysis that summarizes the patterns across time has been popularly used in a variety of disciplines, such as business[ 1 ], financial market[ 2 ], economics[ 3 ] and epidemics or mortality[ 4 – 7 ]. Trend analysis helps to estimate the quantities of current or previous events and their variability or uncertainties in different time points. It is also the foundation for prediction and projection after analyzing the significance of time and relationships with other predictors[ 8 – 10 ]. For national surveys, certain trends have been studied to show the progress or deterioration in public health and health care[ 11 ]. These trends provide important clues for the healthcare professionals to understand the unmet needs for care and the magnitudes of health problems. The comparison of multiple trends allows us to prioritize the issues and allocate resources[ 4 , 12 ]. If well conducted, projections can be made to further prepare incoming challenges to health systems[ 8 , 9 ].

However, there are certain issues arising if taking this approach. First, the adjustment of survey design requires researchers to assign appropriate weights and specify survey sampling units and strata[ 13 ]. The identification of the necessary variables requires extra attention and expert knowledge. Second, the adjustment of survey design also limits the options of research tools[ 14 ]. The automatic procedures developed for time series data or repeated surveys are not applicable concerning survey design[ 1 ]. Linear methods, such as generalized linear models and principal component analysis, remain useful for surveys to generate nationally representative statistics[ 14 , 15 ].

Third, the access to the data may be restricted. For example, some of the Statistics Canada data products can be accessed only through the Research Data Centres (RDC) for academic researchers, such as the Canadian Health Measures Survey (CHMS)[ 16 ]. Physical restrictions may prevent complicated or exhaustive research protocols from being conducted for researchers outside Statistics Canada or other collaborating agencies. Fourth, the outcomes analyzed in national surveys are often limited to individuals’ interests. There are many published studies conducted trend analysis of the CHMS data but only limited numbers of variables are taken as target for analysis, especially hypertension and obesity related factors[ 17 – 23 ]. Even if trends are studied by data holders or affiliated researchers, important issues may remain unanswered. For example, the extensive review of environmental chemicals by Health Canada is not informative because statistics are listed by cycle without testing the significance of time trends or association with other contextual factors[ 24 – 26 ]. This needs to be addressed because effective use or extensive application of trend analysis to national surveys may lead to more efficient biomonitoring[ 11 ] and better identification of unexpected disease trends[ 17 ].

Four, trend analysis may impose challenges to computing resources[ 27 , 28 ]. The large numbers of variables in national surveys may limit the use of this method if not well planned. Lastly, there may not be sufficient incentive for academia, especially the researchers mainly funded by research grants, to innovate toward novel objectives in the long run[ 29 ]. Trend analysis with national surveys requires exhaustive research on documentation and survey method beforehand. There is no immediate benefit by studying variables other than the outcomes that are related to or can lead to research funding.

To address these issues that may be encountered while conducting trend analysis with national surveys, this study aims to 1) propose a framework of trend analysis for all variables in national surveys developed based on the CHMS data, 2) test the feasibility of trend analysis with all CHMS variables using computing resources available to most researchers, 3) summarize the results of the research framework and compatibility with previous studies, and 4) describe some of the obstacles and issues that may be encountered if applied to other surveys.

There were several major steps designed to execute this framework with the CHMS data after reviewing the data structure, data dictionaries, the CHMS User Guide[ 30 , 31 ] and the CHMS Cycle 1 to 8 Content Summary[ 32 ]. This framework was applied to the CHMS data to generate a customized research flowchart in Fig 1 . First, all variables were imported from data files and screened for basic characteristics, including file names, variables of weights, bootstrap weight files to be merged, maximal and minimal values, responses and variable types (continuous or categorical). For the CHMS variables, the maximal values were important for data cleaning because the missing values were always coded with values far exceeding the observed values[ 30 , 31 , 33 ]. The values ending in 4, 5, 6, 7, 8, and 9 might represent “values higher than limits of detection”, “values less than limits of detection”, “not applicable”, “don’t know”, “refusal” and “not stated”[ 30 , 31 , 33 ]. For other surveys, missing values might be represented with certain values[ 34 ] or be coded with reserve values, such as -1 to -3[ 35 ]. To prevent computer memory from being exhausted, the data sets were always removed from the memory if unused.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0200127.g001

Second, user-defined summary variables were be generated once data was stored for cleaning. The summary variables remained blank at this stage and could be the summaries of medication use, biomarker abnormality, or numbers of chronic conditions, depending on the research objectives. After these two steps, an exhaustive list of the CHMS variables was created. Original and derived variables were listed together and could be important indicators of data processing quality. An illustration of the variable list was shown in Table 1 .

https://doi.org/10.1371/journal.pone.0200127.t001

Third, the CHMS data were cleaned based on the reserve values ending in four to nine[ 30 , 31 , 33 ]. The problem particular to biomarker data was that there were values larger or less than the upper or lower limits of detection. Health Canada imputed the values less than the limits of detection with half of the limits of detection[ 24 – 26 ]. In addition, Health Canada excluded the variables with more than 40% of subjects having values less than limits of detection from analysis[ 24 – 26 ]. In contrast, there were currently no official guide to impute values larger than the upper limits of detection and were tentatively imputed with 110% of the upper limits of detection.

Fourth, the summary variables or the derived ones needed to be recoded or calculated after data cleaning. For example, the summary variables of medication use included the use and the numbers of prescription drugs for cardiovascular conditions. This needed to be derived from the drug codes, either Anatomical Therapeutic Chemical (ATC) Classification System or American Society of Health-System Pharmacists (ASHP) drug codes[ 36 ]. Another example was that the chronic conditions reported in the CHMS could be further simplified or summarized in the numbers of chronic conditions diagnosed. Abnormality of disease biomarkers could be identified through external information, such as the clinical reference ranges used by health professionals[ 37 , 38 ]. The numbers of abnormality in biomarkers could be derived after data labeling. Certain secondary biomarkers, such as the estimated creatinine clearance that is used to evaluate kidney health[ 39 , 40 ], could also be derived after data cleaning.

In addition, some of the original variables needed to be made consistent across the CHMS cycles. The inconsistency arose for a variety of reasons, such as the changes in the measurement sample (serum or plasma), whether subjects fasted or not, and categorization of continuous variables. For example, the level of glucose was measured with plasma in the CHMS cycle 1 and with serum in the other cycles. In cycle 3 and 4, glucose was only quantified with fasted subjects. The glucose measurement with serum or plasma could be taken compatible[ 41 ] and could be recoded to the same variable. However, the fasted glucose levels had different diagnostic values from those not fasted and needed to be distinguished[ 42 – 44 ]. Therefore, glucose measured with serum or plasma among fasted and non-fasted subjects were recoded to two variables that represented fasted glucose in cycle 3 and 4 and non-fasted in cycle 1 and 2.

Fifth, some of the summary or derived variables needed to be merged to other data sets to obtain useful statistics. For example, the file of medication use in the CHMS cycle 3 was not assigned survey weights and needed to be merged with the household or other data files to understand issues such as prevalence of drug use or numbers of prescription drugs. The other example was that the information on non-environmental biomarkers in cycle 3 was stored in a stand-alone data set with identification numbers that could be used for data merging. In such cases, the summary variables of medication or abnormality in clinical biomarkers were generated in respective data files and merged to household data files for inference.

Sixth, descriptive or analytical study of all CHMS variables could be conducted. In this study, trend analysis was performed with the CHMS cycles in continuous scales as the only predictor to understand whether there were significantly increasing or decreasing trends across cycles. It was also possible to add more predictors that were important for researchers, such age, sex and provinces. Continuous and binary outcomes were analyzed with linear and logistic regression respectively. The sample sizes, model fit statistics, p values of predictors and variance inflation factors of all predictors were obtained. However, there were several issues to be dealt with for the adjustment of survey design. The sample sizes should be sufficient relative to the primary sampling units. For the CHMS, the sampling units were the cities of clinical visits[ 30 , 31 , 33 ]. The numbers of unweighted sample sizes should satisfy the vetting rules administered by Statistics Canada, which varied by survey and analytical method. The collinearity issue could be assessed between predictors[ 45 ]. To avoid memory overload and increase computation efficiency, only necessary variables were loaded for regression analysis. Lastly, the results were reorganized for vetting and release. The trends were plotted against the CHMS cycles along with the necessary summary tables designated for release vetting by the RDC analyst.

Age[ 46 ] and blood pressure[ 47 ] that had official statistics released were the examples of trend analysis using the CHMS data. The trends were illustrated in relative values compared to the mean values in the CHMS cycle 1. The 95% CIs (confidence intervals) were plotted as shade areas. The details in the blood pressure measurement could be found elsewhere[ 48 , 49 ]. The significance of time trends was confirmed if there was significant association with the CHMS cycles in continuous scale based on linear regression adjusting for survey design. The association with body mass index (BMI) was also tested with linear regression, while age in years, sex, household income in Canadian dollars, and educations in four categories (less than secondary school education, secondary school education, some post-secondary, and post-secondary graduation) were controlled. BMI was calculated as weight in kilograms divided by height in meters squared[ 15 , 50 ]. This study was conducted at the Research Data Centre (RDC) at McGill University (Montréal, Québec, Canada). The computer at the RDC was equipped with Intel i7 3070 CPU (central processing unit, 4 cores 8 thread), 16 GB RAM (Random-access memory), 128 GB SSD (solid state disk) and an operating system, Window 7 Professional 64 bit (Microsoft Corporation, Seattle, USA). Data processing and analysis were conducted with R (v3.20)[ 51 ] and RStudio (v0.98.113)[ 52 ]. Biomarkers were the variables that were identified in the CHMS Cycle 1 to 8 Content Summary[ 32 ]. This Summary also defined environmental biomarkers that were the chemicals that could be detected in human specimens or living spaces Statistics Canada, 2015 #451}. P values, two-tailed, less than 0.05 were considered statistically significant. The processing time was reported to help researchers understand the complexity of trend analysis using national surveys.

Data processing and analysis

There were 26557 original variables in 79 data files released before March 2017. In 32 data files, 16064 variables were related to bootstrap weights only. There were 19212 variables created to summarize data or derived to represent important secondary outcomes for future projects. Using a typical desktop computer at McGill RDC, the processing time of each major step was estimated in Fig 1 . First, the data were imported from STATA format and then stored in R data format. Data importation, storage and screening took less than five minutes to finish. In the third step, the cleaning of all original variables took less than 30 minutes. However, the creation of the summary measures or derived secondary outcomes in the fourth step, such as the numbers of chronic conditions, medication use, and abnormality in biomarkers, was time-consuming. The processing time could be up to two days. At least two factors were contributing to the long processing time. The first factor was that efficient variable-wise calculation was not applicable. Depending on the nature of derived variables, there might be subject-based operation and each observation needed to be screen, for example, for the numbers of cardiovascular or diabetes medication for each individual. The other factor was due to time spent on loading data to memory and writing processed data back to disk.

In the fifth step, the summary or derived variables that needed to be linked to or reproduced in other data files, such as the information on medication use and biomarker summaries, were merged to destination files. For example, the summary of medication use needed to be merged to the household data set and used with appropriate bootstrap weights to obtain nationally representative statistics. This took less than one hour to finish. Sixth, trend or regression analysis with and without the adjustment of other predictors took less than one day to finish for all original or derived variables. The predicted values of all CHMS variables could also be calculated within one day. Lastly, selected trends and summary tables were produced for vetting and release from the RDC within 10 minutes. This research framework took less than four days to screen and analyze all CHMS variables.

Characteristics of the CHMS cycles and Canadians

The summary of the CHMS data and the population characteristics were shown in Table 2 . The cycle 3 had the most numbers of variables and many of them were ever repeated in other cycles. There were cycle-4 variables to be released after April 2017. In cycle 2, there were more biomarkers than in any others. Because of the large numbers of biomarkers in cycle 2 and 3, there were variables designed to label limits of detection for all subjects.

https://doi.org/10.1371/journal.pone.0200127.t002

The numbers of Canadians increased over time, from 29 to 32 million between cycle 1 and 4. About half of them were female. The proportion of females may not be different from that obtained with other data sources[ 53 ]. The minimal ages were three years in cycle 1 and six in cycle 2 to 4. The maximal ages were 79 for all cycles. The mean age remained similar and might not be different from the official statistics, which described age by median values[ 46 ]. The ranges of blood pressure might also be similar to those published based on the same data[ 47 ], while Canadians of all ages were included in this study. In Fig 2 , the trends of age, mean arterial pressure, and systolic and diastolic blood pressure were shown along with their 95% confidence intervals (CIs) compared to the first measures in the CHMS. None of the trends was significantly associated with the CHMS cycles (p> 0.05 for all). Age and blood pressure were significantly associated with BMI while controlling for age, sex, education and household income (p <0.05 for all).

*significantly associated with body mass index (p<0.05).

https://doi.org/10.1371/journal.pone.0200127.g002

Summary of the trends in the CHMS data

In Table 3 , the findings of the trend analysis were summarized. In the first row, the numbers of the CHMS variables that had been repeatedly measured were listed. There were 519 variables measured in CHMS cycle 1 to 4. The rates of change of BMI from cycle 1 to 4 were listed. There were 429 variables significantly associated with the CHMS cycles from one to four and 86 of them were biomarkers identified by Statistics Canada (p<0.05 for all). There were 1099 variables significantly associated with BMI and 152 of them were biomarkers. There were 20 and 26 variables respectively increasing and decreasing for more than 10% in three time intervals from cycle 1 to 4. There were 52 and 68 biomarkers observed to respectively increase or decrease once for at least 10% from cycle 1 to 4. Compared to the average growth rates of BMI, 0.2% per cycle, there were 130 biomarkers increasing more rapidly and 22 of them were non-environmental biomarkers.

https://doi.org/10.1371/journal.pone.0200127.t003

There are large numbers of the CHMS variables and biomarkers increasing or decreasing at high rates. The importance of these trends to public health and wellbeing are not clear because current rate of investigating and publishing the trends of the CHMS variables is not satisfying. There were less than ten trends of the CHMS variables published between 2015 and 2017 including those only considering selected populations[ 48 , 54 , 55 ]. It can take more than ten years to have a comprehensive understanding in the trends of the biomarkers or physical activities or other variables, given the large numbers of variables in national surveys. Currently the CHMS data have been mostly used as a novel data source[ 12 , 18 – 23 , 56 , 57 ], rather than a continuous effort to monitor population health. Only several outcomes have been studied continuously among selected populations[ 48 , 54 , 55 ], in addition to the biomonitoring activities by Health Canada[ 24 – 26 ].

This research framework of trend analysis customized to the CHMS data is highly feasible with computing resources available to most researchers. Scaling up trend analysis to all variables in national surveys has several advantages. In the first place, the automated data cleaning system is effective and efficient. It takes less than 30 minutes to clean all 79 files from the CHMS cycle 1 to 4. The results of data cleaning are examined based on parameters such as the maximal or minimal values to ensure appropriate quality for subsequent trend analysis. Another advantage is that the visualization of trends is easy to understand and useful to prioritize biomarkers or variables for evaluation. In this study, the trends of blood pressure is plotted with the BMI trend to contrast the different patterns. We are applying this method to other variables to find unexpected trends. Moreover, certain types of data errors can also be easily highlighted with the trends. For example, the measurement unit of blood fibrinogen is mislabeled and leads to more than 10-fold decrease in the levels after the CHMS cycle 2 (personal communication with Statistics Canada). The trends with the highest and lowest rates of increase or decrease are easy targets for data quality examination.

Finally, this framework of trend analysis can be supplemented with regression analysis, prediction and projection subsequently. Multiple regression for all CHMS variables to identify the significance of BMI and socioeconomic status has been tried and proven realistic. Predicted values are retrieved to understand the trends least explained by BMI and socioeconomic status (statistics not requested for release). The CHMS has also been used for the projection of obesity trends[ 10 ] and projection is also possible.

Limitations

However, there are several limitations to the research framework. First, there may be other data or documentation errors not identified. The data and documentation accuracy of several of the trends of the largest relative magnitude of change have been confirmed (personal communication with Statistics Canada). There may be other errors that cannot be identified with trend analysis. The other issue is that the imputation method for right- or left-censoring can be improved. Health Canada imputes censored environmental chemicals according to the limits of detection and proportions of subjects within the limits[ 24 , 26 , 58 ]. Other advanced methods may be tried to take other contextual factors into consideration[ 59 , 60 ]. In fact, it is unclear whether the proportions used by Health Canada are based on weighted or unweighted statistics[ 24 , 26 , 58 ]. This study uses unweighted proportions to exclude the variables from analysis.

Furthermore, the codes have been written inside the RDC and suffered from significant time and resource constraints. The research framework will be structured into an R package for application to other major surveys and research purposes. There are several improvements expected for the implementation. For example, the evaluation of data products can be customized and made interactive. The method to create a list of variable characteristics to be extracted is related to the research hypothesis and should be made flexible for other projects. The introduction of external information to create or derive new variables as predictor or outcome can be improved. We are introducing the reference ranges for clinical or disease biomarkers[ 37 , 38 ] to further interpret clinical data and population health status. A system that describes the relationships between variables to infer information between them will be useful for sequential questions that study complicated status, such as disease history or evolution of life events. We are also considering incorporating imputation of missing information into the research framework[ 60 ].

Extension to other surveys

This research framework can be extended to other major surveys with similar data structure, variable naming systems, missing value identification strategies and sampling frames, especially the Canadian Community Health Survey[ 48 , 56 ]. For other major surveys that provide cleaned data[ 61 ] or do not use bootstrap weights[ 35 ], it requires minimal revision to replicate this research framework to conduct trend analysis for all variables. The automated process for visualization of trend analysis is suggested for researchers to look for neglected trends and for survey administrators to search and correct data errors that can be demonstrated with trends of extreme rates of change across cycles or time points.

Declaration

Ethics review.

This secondary data analysis was approved by the ethics review committee at the Centre Hospitalier de l’Université de Montréal.

Acknowledgments

The analysis presented in this paper was conducted at the Quebec Interuniversity Centre for Social Statistics, which is part of the Canadian Research Data Centre Network (CRDCN). The services and activities provided by the QICSS are made possible by the financial or in-kind support of the Social Sciences and Humanities Research Council (SSHRC), the Canadian Institutes of Health Research (CIHR), the Canada Foundation for Innovation (CFI), Statistics Canada, the Fonds de recherche du Québec—Société et culture (FRQSC), the Fonds de recherche du Québec—Santé (FRQS) and the Quebec universities. The views expressed in this paper are those of the authors, and not necessarily those of the CRDCN or its partners[ 16 ].

• 1. Hyndman RJ, Khandakar Y. Automatic time series for forecasting: the forecast package for R. Monash University, Department of Econometrics and Business Statistics, 2007.
• 2. Thomsett MC. A Technical Approach To Trend Analysis: Practical Trade Timing for Enhanced Profits: Pearson Education; 2015.
• 3. States United. Internal Revenue Service. Research Division. Trend Analysis and Related Statistics: Department of the Treasury, Internal Revenue Service; 1988.
• View Article
• PubMed/NCBI
• Google Scholar
• 11. Committee to Review the Department of the Interior's Biomonitoring of Environmental Status and Trends Program, Board on Environmental Studies and Toxicology. A Review of the Biomonitoring of Environmental Status and Trends Program: The Draft Detailed Plan. Committee to Review the Department of the Interior's Biomonitoring of Environmental Status and Trends Program, editor. Washington, D.C.: National Academy Press; 1995.
• 13. Statistics Canada. Survey Methods and Practices. Statistics Canada, editor. Ottawa, ON: Statistics Canada,; 2010.
• 16. The Quebec Inter-University Centre for Social Statistics (CIQSS). Statistics Canada Data Montreal, QC: The Quebec Inter-University Centre for Social Statistics (CIQSS),; 2017 [cited 2017 Feb 1]. Available from: https://www.ciqss.org/en/statistics-canada-data .
• 24. Canada Health. Third Report on Human Biomonitoring of Environmental Chemicals in Canada. Ottawa, ON: Health Canada,; 2015. Available from: http://www.hc-sc.gc.ca/ewh-semt/alt_formats/pdf/pubs/contaminants/chms-ecms-cycle3/chms-ecms-cycle3-eng.pdf .
• 25. Health Canada. Second Report on Human Biomonitoring of Environmental Chemicals in Canada. Health Canada, editor. Ottawa, Ontario: Health Canada,; 2013.
• 26. Health Canada. Report on Human Biomonitoring of Environmental Chemicals in Canada. Ottawa, ON: Health Canada,; 2010. Available from: http://www.hc-sc.gc.ca/ewh-semt/alt_formats/hecs-sesc/pdf/pubs/contaminants/chms-ecms/report-rapport-eng.pdf .
• 30. Statistics Canada. Canadian Community Health Survey (CCHS) Annual component User guide 2014 and 2013–2014 Microdata files. Ottawa, ON: Statistics Canada, 2015.
• 31. Statistics Canada. Canadian Health Measures Survey (CHMS) Data User Guide: Cycle 2. Ottawa, ON: Statistics Canada, 2013 April. Report No.
• 32. Statistics Canada. Canadian Health Measures Survey (CHMS) Content summary for cycles 1 to 8 In: Statistics Canada, editor. Ottawa, ON: Statistics Canada,; 2015.
• 33. Statistics Canada. Canadian Health Measures Survey (CHMS) Data User Guide: Cycle 1. Ottawa, ON: 2011 April. Report No.
• 34. Health and Retirement Study, RAND HRS Data File (v.P) public use dataset. In: U01AG009740). PadbtUoMwfftNIoAgnN, editor. Ann Arbor, MI2016.
• 37. Royal College of Physicians and Surgeons of Canada. Clinical laboratory tests—Reference values. In: values Clt-R, editor. 10–2015 ed. Ottawa, ON: Royal College of Physicians and Surgeons of Canada,.
• 38. Longo D, Fauci A, Kasper D, Hauser S, Jameson J, Loscalzo J. Harrison's Principles of Internal Medicine, 19th Edition: McGraw-Hill Education; 2015.
• 39. Eaton D, Pooler J. Chapter 3. Clearance. In: Eaton D, Pooler J, editors. Vander’s Renal Physiology 8ed. New York: McGraw-Hill; 2013.
• 44. World Health Organization. Use of Glycated Haemoglobin (HbA1c) in the Diagnosis of Diabetes Mellitus: Abbreviated Report of a WHO Consultation. Geneva, Switzerland: World Health Organization, 2011.
• 46. Milan A. Age and sex structure: Canada, provinces and territories, 2010. Ottawa: Statistics Canada. 2011.
• 47. Statistics Canada. Blood pressure of Canadian adults, 2009 to 2011. Ottawa, ON: Statistics Canada, 2012 Contract No.: 82-625-X.
• 51. R Development Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2016.
• 52. RStudio Team. RStudio: Integrated Development for R. Boston, MA: RStudio, Inc.; 2016.
• 53. Urquijo CR, Milan A. Female population. Women in Canada: A Gender-Based Statistical Report Component of Statistics Canada Catalogue No 89-503-X. 2011.
• 58. Health Canada. Second Report on Human Biomonitoring of Environmental Chemicals in Canada. Ottawa, ON: Health Canada, 2013.
• 61. RAND Corporation. RAND HRS Data Files, supported by NIA and SSA Santa Monica, CA: RAND Corporation,; 2016 [updated September 2016; cited 2016 Nov 29]. Available from: http://www.rand.org/labor/aging/dataprod/hrs-data.html .

• A/B Monadic Test
• A/B Pre-Roll Test
• Key Driver Analysis
• Multiple Implicit
• Penalty Reward
• Price Sensitivity
• Segmentation
• Single Implicit
• Category Exploration
• Competitive Landscape
• Consumer Segmentation
• Innovation & Renovation
• Product Portfolio
• Marketing Creatives
• Advertising
• Shelf Optimization
• Performance Monitoring
• Better Brand Health Tracking
• Ad Tracking
• Trend Tracking
• Satisfaction Tracking
• AI Insights
• Case Studies

quantilope is the Consumer Intelligence Platform for all end-to-end research needs

What Is Trend Analysis in Research? Types, Methods, and Examples

Trends are everywhere. They are central to how businesses craft their product development, marketing, and dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268031">social media strategies, and how consumers go about purchasing decisions.

Trends are sometimes driven by dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268011">external factors (like a shortage of a certain product that creates a trend for something new), and other times trends are driven by internal consumer wants/needs (like plant-based dairy alternatives). Businesses that pay attention to and understand current/evolving trends (through dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis ) are able to use dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268028">informed dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268003">decision dropdown#toggle" data-dropdown-menu-id-param="menu_term_289268003" data-dropdown-placement-param="top" data-term-id="289268003">-making in their operations. This article looks at different dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">types of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis , how to conduct it, and how to act on emerging trends to stay ahead of the competition.

Table of contents

• What is trend analysis?
• Types of trend analysis 
• How to conduct trend data analysis
• How to use trends analysis for virtually any type of research 
• Example of trend analysis in market research
• Advantages and disadvantages of trend analysis
• Use quantilope for automated trend analysis

What is dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis ?

There’s a reason why people talk about ‘the latest trends' the way they might also talk about the ‘latest’ news or ‘latest’ developments. There’s also a reason why people talk about trends evolving over time. Trends can be temporary - around for a while and then gone in a flash, as is often the case with fashion or some hairstyles (unless they make a comeback...like flare jeans and bucket hats). Meanwhile, other trends might gain momentum slowly and grow steadily over time, like tech usage or certain diets.

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">Trend analysis is the process of using dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267999">historical data as well as current dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268001">data sets to determine how consumers behave and how businesses react; the same is true of the inverse: how businesses behave and how consumers react. dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">Trend analysis focuses on dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268004">market trends over a dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268006">period of time and can be used as an ongoing resource to keep ahead of market changes. Whether it’s used in the dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268010">short term or the long term, dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis can reveal changes in consumer needs as well as changes in industry activity. These aren’t always going to be huge, industry-wide trends; they can be smaller ones too - such as small dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268021">fluctuations in consumer loyalty or satisfaction with a particular product, or dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268042">downtrends and dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268041">uptrends in certain product usage. Businesses use their dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis findings to act on emerging trends as well as to predict dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268039">future trends and plan dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268032">new products or marketing activity accordingly. Back to table of contents  

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">Types of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

There are various dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">types of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis available through market research. Below we’ll touch on a few of the most popular types that can guide businesses’ dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268003">decision making for different needs .

Consumer dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

This dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">type of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis relates to how consumers behave, think, and purchase within a certain sphere or market landscape. It could uncover consideration and usage of a product or service, consumer behavior in a specific product category, consumers’ dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268031">social media usage, or how consumers feel about political, social, or environmental issues. Information gathered from consumer dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis helps businesses leverage those consumer preferences in their current business operations and identify new growth opportunities.

Competitor dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Knowing where competitors are winning and losing is crucial information to feed into business decisions. Analyzing how competitors have performed at certain points in time, such as the launch of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268032">new products or advertising campaigns, reveals how positively the target market reacts to those types of business activities. This dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">type of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis helps identify strategies that will encourage consumers to choose your business over competitors, as well as to find new opportunities where competitors are weak.

Historical dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Looking at past dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268000">data points and tracking how consumer attitudes, consumer behaviors, or industry activities have changed in relation can provide valuable context for dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268013">future events . Say for example you sell beauty products and you’ve seen the popularity of vitamins in body cream grow over the past two years; this is a good indication that the trend will continue, which will help shape dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268032">new product development and future marketing messages.

Temporal dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Temporal dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis looks at a specific dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268006">period of time to see how consumer trends have changed in that dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268018">time frame alone. You could take one or more periods of time and compare them, or even analyze based on dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268009">seasonality (e.g. summer, the holiday dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268009">season ). This dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">type of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis helps identify trends at a set time which can be helpful when planning inventory stock, pricing strategies, or product promotions for similar time periods in the future.

Geographic dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Geographic dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis looks at changes within geographical locations and compares them with each other. For example, how have skincare preferences evolved in Asia, and how does this compare with preferences in North America? Trends in one region could give clues as to how trends will develop in another - especially today with global dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268031">social media platforms like Instagram and TikTok that can spread geographical trends in record speed. This dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">type of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis is useful for international businesses looking to shape their offer in each location they operate in.

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268040">Demographic dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Knowing your target market is essential to running a successful business. You need to keep tabs on what your consumers want and need, and how those differ based on factors like age, gender, region, etc. Older consumers may have different dietary needs than younger ones; the same goes for cosmetics, footwear, haircare, technology, and so on. This dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268012">type of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis is great for understanding how a particular dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268040">demographic group has changed over time so brands can appeal to that audience with the right communication and product portfolios.

Economic dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Inflation and the general cost of living are examples of economic trends that give businesses a good idea of current consumer buying power and their likely willingness to spend. Economic trended dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268001">data sets are typically available publicly, along with a company’s own internal dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268008">financial statements . This type of data is helpful to reference when setting new price points or making upcoming production decisions.

Technological dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

Technology is continuously evolving, and there’s no doubt it will continue to do so. In recent years alone it seems to be evolving faster than ever with things like self-driving cars, virtual reality, and the rise of AI. Technological dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis empowers organizations to make dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268028">informed decisions and gain a competitive advantage. Businesses can use technology trends to operate more efficiently, foster new innovations, and to understand consumer expectations better. Back to table of contents  

How to conduct dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268025">trend dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268005">data dropdown#toggle" data-dropdown-menu-id-param="menu_term_289268005" data-dropdown-placement-param="top" data-term-id="289268005"> analysis  

Below are a few simple steps to getting started with your trend analysis research study: 

1. Define your goals

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268004">Market dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend dropdown#toggle" data-dropdown-menu-id-param="menu_term_289267998" data-dropdown-placement-param="top" data-term-id="289267998"> analysis requires a clear starting point and a clear end point. In other words, what do you know already and what do you hope to find out? The latter will determine your end goal(s).

Your goals will guide your dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis throughout each stage - from initial survey setup to final analysis. When you start looking through your dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268001">data set , your end research goal will help you focus on the trends that actually impact your business.

2. Invest in regular dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trends analysis

Identifying trends doesn’t happen overnight. Trends appear over continuous dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268018">timeframes - known as ‘waves’ in trend research. You need to collect data on an ongoing basis to find those trends, and the best way of doing so is setting up a consistent research tracker. Monthly, quarterly, twice-yearly, or annual tracking surveys are some of the most commonly-used cadences to identify trends over time. The frequency of your tracker will depend on how dynamic your industry is; CPG product preferences can change all the time whereas something like home/car insurance may be less wavering.

3. Find an easy-to-use survey tool

An intuitive survey tool - like an online research platform , can speed up your dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268005">data analysis process to act on insights faster. Easy-to-use survey tools offer things like research expert consultation, drag & drop modules, automated advanced methods , real-time reporting, and easily designed dashboard reports that can be shared around without the risk of version control. 

4. Identify your sample

For quality data, you need to find the right people and ask the right questions. This means launching a survey among respondents who accurately reflect your target audience and asking questions that relate to your previously-defined goals; the right survey tool will make sure you can achieve both of these by offering things like survey templates and panel agnostic capabilities.

5. Field and analyze your data

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268005">Data analysis will highlight trends that arise from consumer behavior, competitive behavior, or general industry behavior. A good dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268005">data analysis platform will allow you to review results in real-time, as respondents complete your survey - rather than having to wait until the end of fieldwork for a data processing team to send over a final cross-tab file. As you review your data, you can cut dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268020">metrics by different parameters and dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268040">demographics to understand various trend perspectives. Your final data will go into a dashboard or report to share with dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268016">stakeholders for next steps.

6. Act on your findings

Once you’ve analyzed and reported on your trended data findings, it’s time to take action. This might mean immediate action, like putting a dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268032">new product into market, or waiting for another wave of data to confirm a suspected trend. Regardless, the insights you’ve gained from your dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis can feed into future business dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268003">decision-making to stay ‘on trend’ and ahead of competitors. Back to table of contents  

How to use dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trends analysis for virtually any type of research

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">Trend analysis can be used to uncover almost any trends. Above we’ve already mentioned the benefits in exploring trends amongst consumers, competitors, and dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268040">demographics , along with using dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis to uncover geographic, economic, and technological changes. Other use cases of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis include:

Customer satisfaction. Understanding satisfaction levels with regards to a product or service, and how this relates to a brand’s activity or competitor performance. Part of this measurement might be tracking a brand’s NPS score over time.

Employee satisfaction. Identifying how employee turnover or loyalty relates to the company ethos or other factors.

Customer spend. Tracking how different customer types allocate budget to a product over time reveals trends in disposable cash levels as well as their willingness to spend. This feeds into determining dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268032">new product price points and planning dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268032">new product offers.

Financial dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268021">fluctuations and forecasts. Pinpointing where sales have peaked or dipped, and whether there has been an dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268034">upward trend or downward trend since then, provides crucial information on when businesses should explore new opportunities. It also helps predict how business activity will shape future growth.

The customer experience. Part of understanding your target audience means appreciating how their experience of your brand correlates to prevailing trends. This is separate from overall satisfaction; a customer might be satisfied with the end product or service but not the process in finding or purchasing it. Back to table of contents  

dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268027">Example of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis in market research

Companies can use dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis to inform their spend, product development, advertising, and just about all other areas of business operations. For example, quantilope runs an annual mattress tracking study that identifies trends around in-store vs. online mattress purchasing, direct-to-consumer mattress buying, the popularity of certain mattress brands, and so on. Over the past few years, consumers’ shopping experiences have shifted heavily online - and our mattress tracker showed that mattresses were no exception. Between 2019 and 2020 onward, the study showed a significant dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268034">upward trend in online mattress purchasing.

Advantages and disadvantages of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis

The advantages of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trends analysis include:

Informing a business’ overall strategy. Trends are pervasive, influencing customer behavior and attitudes across many aspects of their lives. Using these insights can form a strong foundation for the development of a variety of business activities - from product development to marketing campaigns.

Capturing dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268011">external factors that influence a market, such as political and economic events. These external events could have drastic impact on a business if they go unnoticed. A lot of times businesses will think about aspects of the market that are directly related to their activities, but external trends can also have major impact on consumers' purchasing power. 

Providing ongoing feedback for effective future planning. Rather than one-off studies around a specific topic or event, trended data is what many dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268016">stakeholders ultimately  rely on when making important business decisions.

Some potential disadvantages of dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis include:

A lack of the full picture. Sometimes dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis studies will require additional deep-dive research to really understand what’s going on in your market or among your consumers. dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">Trend analysis tells you what people or markets are doing, but not always necessarily why.

It requires time. dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267999">Historical data is the only real way to see how trends are shaping up over time, so businesses need to dedicate time and patience into a full trend study to ensure that suspected trends aren’t just periodic ‘blips’.

Because it requires time, it’s not always cheap. Since dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis requires several waves, that means you have to pay for panel respondents each time you run the study, along with any other fieldwork costs your chosen survey provider requires. The key is finding a platform that doesn’t come with a lot of added fees and offers an affordable tracking solution. Back to table of contents

Use quantilope for automated dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis  

quantilope offers intuitive and affordable dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis through its tracking solutions.

Choose between a category tracker or quantilope’s new brand health tracking approach that uses industry-praised concepts such as category entry points and Mental Availability . Either way, quantilope users will start with the option to customize a pre-built survey template or build their own tracking study by scratch. Building your tracker is made easy through a library of pre-programmed questions and advanced dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268033">methodologies that you simply drag & drop into your survey builder. The platform even offers an AI co-pilot, quinn , to assist you in your survey creation, analysis, and reporting processes. Findings are available in real-time, with the option to start building report charts long before fieldwork wraps up. Once it does, all charts are automatically updated with final data and statistical testing. Cut the data any way you like, by any other variable within your dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289267998">trend analysis survey. Store all final charts in the reporting tab of the platform to use in a final dashboard deliverable, which is shareable with dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268016">stakeholders through a single link.

Subsequent waves of your dropdown#toggle" data-dropdown-placement-param="top" data-term-id="289268025">trend data research can be set live on the platform with a few clicks of a button, as often as you choose. Trended data is automatically added to existing charts in real-time, so you never have to go back to square one.

For more on how quantilope can help your business ahead of trends (and the competition), get in touch below!

Get in touch to learn more about trend analysis with quantilope!

Related posts, quantilope & organic valley: understanding consumer values behind behaviors, quantilope & wire webinar: solving the research dilemma with ai, a full year of better brand health tracking in the soda category, non-probability sampling: when and how to use it effectively.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 25 June 2020

Analyzing trend and forecasting of rainfall changes in India using non-parametrical and machine learning approaches

• Bushra Praveen 1 ,
• Swapan Talukdar 2 ,
• Shahfahad 3 ,
• Susanta Mahato 2 ,
• Jayanta Mondal 2 ,
• Pritee Sharma 1 ,
• Abu Reza Md. Towfiqul Islam 4 &
• Atiqur Rahman 3  

Scientific Reports volume  10 , Article number:  10342 ( 2020 ) Cite this article

49k Accesses

226 Citations

19 Altmetric

Metrics details

• Climate sciences
• Natural hazards

This study analyzes and forecasts the long-term Spatio-temporal changes in rainfall using the data from 1901 to 2015 across India at meteorological divisional level. The Pettitt test was employed to detect the abrupt change point in time frame, while the Mann-Kendall (MK) test and Sen’s Innovative trend analysis were performed to analyze the rainfall trend. The Artificial Neural Network-Multilayer Perceptron (ANN-MLP) was employed to forecast the upcoming 15 years rainfall across India. We mapped the rainfall trend pattern for whole country by using the geo-statistical technique like Kriging in ArcGIS environment. Results show that the most of the meteorological divisions exhibited significant negative trend of rainfall in annual and seasonal scales, except seven divisions during. Out of 17 divisions, 11 divisions recorded noteworthy rainfall declining trend for the monsoon season at 0.05% significance level, while the insignificant negative trend of rainfall was detected for the winter and pre-monsoon seasons. Furthermore, the significant negative trend (−8.5) was recorded for overall annual rainfall. Based on the findings of change detection, the most probable year of change detection was occurred primarily after 1960 for most of the meteorological stations. The increasing rainfall trend had observed during the period 1901–1950, while a significant decline rainfall was detected after 1951. The rainfall forecast for upcoming 15 years for all the meteorological divisions’ also exhibit a significant decline in the rainfall. The results derived from ECMWF ERA5 reanalysis data exhibit that increasing/decreasing precipitation convective rate, elevated low cloud cover and inadequate vertically integrated moisture divergence might have influenced on change of rainfall in India. Findings of the study have some implications in water resources management considering the limited availability of water resources and increase in the future water demand.

Similar content being viewed by others

The economic commitment of climate change

Heat health risk assessment in Philippine cities using remotely sensed data and social-ecological indicators

Applying machine learning algorithms to predict the stock price trend in the stock market – The case of Vietnam

Introduction.

Rainfall is a key part of hydrological cycle and alteration of its pattern directly affect the water resources 1 . The changing pattern of rainfall in consequence of climate change is now concerning issues to water resource managers and hydrologists 2 . Srivastava et al . 3 and Islam et al . 4 reported that the changes of rainfall quantities and frequencies directly changing the stream flow pattern and its demand, spatiotemporal allocation of run-off, ground water reserves and soil moisture. Consequently, these changes showed the widespread consequences on the water resource, environment, terrestrial ecosystem, ocean, bio-diversity, agricultural and food security. The drought and flood like hazardous events can be occurred frequently because of the extreme changes of rainfall trend 5 . Gupta et al . 6 documented that the amount of soil moisture for crop production is totally determined by the amount of rainfall. The monsoon rainfall plays a vital role for agriculture in India. 68% of cultivated land to the total cultivated land of India is occupying by the rain fed agriculture which supports 60% of livestock population and 40% of human population 7 . Hence, the research on the climate change or most specifically on the changes of rainfall occurrences and its allocation are the most significant way for sustainable water resource management. Therefore, the sustainable development of agriculture in India requires the noteworthy research on the identification and quantification of climate change 7 . Most importantly, a complete understanding of the precipitation pattern in the changing environment will help in better decision making and improve the adapting-capacity of the communities to sustain the extreme weather events.

Nowadays, the water resources have been considered as the key concern for any kinds of development program and planning which includes effective water resource management, food production sector and flood control. The uneven allocation of water supply throughout the country, because of the natural pattern of rainfall occurrence which varies significantly in space and time, is the main hindrance for the effective water resource management in India 8 . The climate change further accelerates this rainfall variability 7 . Consequently, many regions of the country receive huge amounts of rainfall during the monsoon, while others receive very less amount of rainfall and frequently experience the worst reality of water scarcity. Furthermore, the worst consequence of climate change, especially rainfall change, has been experiencing by the agricultural sector of this country. Because the rainfall has not been taking place when it is expected and vice versa 9 . Even, the high winds and hails have been frequently occurred. Consequently, enormous losses to crops have been taken place and the farmers who totally dependent on the agriculture have become devastated 7 . The freakish weather is causing havoc on the farmers community, as a results, the farmers have gone extreme to the committing of suicide. During the period 1995–2014, 300000 farmers have committed suicide in India 10 . Hence, it is noteworthy to evaluate whether there is any trend in rainfall and any pattern in variability 11 .

Therefore, to explore the variability and changes in pattern and existence of trend in rainfall over different spatial horizons have been the key aspects in the study of hydrology, climatology and meteorology worldwide 12 , 13 , 14 , 15 , 16 , 17 . In most studies, researchers have used parametric and non-parametric methods 18 like regression test 19 , 20 , Mann-Kendall test 21 , 22 , Kendall rank correlation test 23 , Sen’s slope estimation 24 , 25 and Spearman rank correlation test 26 . In the present study, the non-parametric test like Mann-Kendall test was applied to detect the trend in rainfall as it is one of the most often applied global methods for trend detection in hydrology, climatology and meteorology 27 , 28 , 29 , 30 , 31 . One of the major reason to use non-parametric tests in the present study is these can be used on independent time series data and are also not much sensitive to outliers 32 . However, the change detection analysis in the climatologic and hydrological data series is the important aspect for the trend analysis throughout the world 33 , 34 .The change detection methods include the Standard Normal Homogeneity Test (SNHT) 35 , Buishand range test 36 and the Pettitt’s test 37 . The trend analysis was carried out before the change point analysis in many researches 32 . This approach can lead to misleading results as the information obtained from change point detection analysis has not been considered for the trend analysis 38 . Hence, Li et al . 36 highly recommended performing the change detection analysis first followed by the trend analysis. The results obtained from this approach are more reasonable and reliable.

In India, several attempts have already been done in the past to detect rainfall trends in regional and national levels 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 . The research on trend analysis has believed to be a useful tool as it provides the significant information about the possibility of the future changes 4 , 47 , 48 . However, multiple studies have conducted trend analysis using MK test and reported that India as a whole has not recorded significant trend of increase or decrease in annual average rainfall 7 , 46 , 49 . Many researchers identified significant trend in long-term rainfall in India 2 , 44 , 50 , 51 , 52 , 53 , while the significant long-term trend has not been detected in monsoon rainfall on a national scale 7 . Patra et al . 54 studied the trend analysis in monthly, seasonal and annual rainfall over Orissa and reported that long-term insignificant negative trend was detected in annual and monsoonal rainfall, while the positive trend was detected in post- monsoon season. But to the best of authors knowledge, most of the researchers applied non-parametric tests to detect and followed by trend analysis. Therefore, to get reliable and reasonable results of trend, we have obtained both approaches like trend analysis before change point and first change point analysis followed by trend detection.

Sen 55 developed the innovative trend analysis that has been utilized for detecting trend in meteorological, hydrological and environmental variables 56 , 57 , 58 , 59 , 60 . The innovative trend analysis has the worldwide applicability over the non-parametric approach 61 like Mann-Kendal test, spearman’s correlation test and Sen’s slope estimation because these non-parametric test are very much sensitive to the distribution assumptions, serial correlation, size of the time series data and seasonal cycle 62 . Von Storch 63 stated that if the data series have statistically significant serial correlation, the MK test can generate no noteworthy trend existence in the data series. Sen 55 stated that the effective, efficient and optimum water resource management needs to identify the trend not only monotonically over time, but also needs to identify trends separately which have the high, medium and low value which can be possible to identify by the innovative trend. In addition, the innovative trend is not sensitive to serial correlation, non-linearity, and size of the time series data and gives the robust and powerful result with less error. Therefore, researchers prefer the innovative trend analysis for detecting trend in hydrological, meteorological studies 18 , 44 , 55 , 64 , 65 . Therefore, in the present study, apart from Mann-Kendall test, the innovative trend was also utilized for detecting trend. In India, very few studies were conducted to detect trend using innovative trend analysis. But to the best of authors’ knowledge, the application of innovative trend to detect seasonal rainfall for whole India would be the first application.

As we have considered detecting change point using MK test before change point analysis, MK test after change point analysis and innovative trend analysis, the planners of water resource management, environmentalist, and hydrologists can have the chance to compare the result of all these techniques and get high precision trend information in annual and seasonal rainfall. Therefore, in this line of thinking, we can conclude that the study has the quite novelty.

The trend analysis facilitates to understand the present and past climatic changes, but the future forecasting is more useful for the planners to execute the proper planning taking into account future changes in climatic variables 66 . To project future information of the climatic variables, in addition to the complicated climate model which works on global scale, regional forecasting could be carried using statistical techniques and machine learning soft computing techniques 67 , 68 . To execute the physical models, large numbers of database, high configured system, advanced technology, technical expert are required. The physical models are both time and money consuming, although they provide highly reliable results. The statistical techniques like auto regressive (AR), moving average (MA), auto regressive moving average (ARMA) and auto regressive integrated moving average (ARIMA) have several limitations such as AR model regresses past values, while MA model utilizes past error as the explanatory variables and ARMA model can perform for stationary time series data 67 . However, recently, the application of artificial intelligence (AI) models like machine learning techniques have gained attention 69 . Unlike physical models, the AI models perform very well as it does not require huge information, but can handle huge and complicated data sets if provided 70 . It can perform in non-stationary time series data 71 . Darji et al . 67 conducted review survey regarding the rainfall forecasting using neural networks and reported that back propagation based neural network performs well to forecast rainfall. In the present study, we utilized multilayer perceptron (MLP) algorithm based neural network for predicting the rainfall. The MLP based ANN works using back propagation algorithm. However, the application of MLP based artificial neural network (ANN) for predicting and forecasting of multifaceted hydrological and climatic phenomena and has gained popularity across the globe as it provides reliable results 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 . The novelty of the present study is that the application of MLP-ANN is not concentrated on the one or more states or overall India rather the present study considers predicting and forecasting rainfall for thirty-four meteorological sub-divisions of India. Therefore, the study will be highly beneficial as it worked on the whole India in micro level.

Therefore, based on the previous literatures, research gaps, the objectives of the present study are: (1) to analyze the variability and trend analysis in the overall annual and seasonal rainfall for thirty-four meteorological sub-divisions; (2) to detect the change point for all sub-divisions; (3) to explore the change point wise rainfall variability and trend analysis; (4) to detect trend using innovative trend analysis; (5) to predict and forecast rainfall up to 2030 for all sub-divisions; (6) to investigate the causes of the changes of rainfall pattern.

Material and Methods

Study area and data source.

The geographical location of India is between 8.4° and 37.6°N latitude and 68.7° and 97.25°E longitude. The whole India can be categorized into four homogenous climatic regions such as North-West India (NW India), North-East India (NE India), Central India and Peninsular India on the basis of the distribution and occurrence pattern of rainfall and temperature 46 . Although the whole country experiences almost all types of climate due to its physiographic location 59 . However, India observes the four seasons, like summer season (January-February), winter (March-May), monsoon (June-September) and post-monsoon (October-December) 82 . The India receives 117 cm rainfall, out of which 80% of rainfall is observed during monsoon month.

The whole Indian region has been sub-divided into thirty-four meteorological subdivisions by the Indian Meteorological Department (IMD) where the climatic parameters like rainfall, temperature, wind speed, humidity have been recorded since 1901 (Fig.  1a ). In this study, we obtained rainfall data for 115 years (1901–2015) from thirty-four meteorological sub-divisions. The collected data was continuous in nature with no missing values.

( a ) Geographical location of the study area; LOWESS curve on annual and seasonal rainfall where figure shows ( b ) annual rainfall, ( c ) winter ( d ) summer ( e ) monsoon and ( f ) post monsoon rainfall.

Method for trend analysis

In this paper we used the Mann-Kendal test 18 , a non-parametric statistical test based on rank system, to detect the trend in long-term rainfall data series. The MK test is mainly used for detecting trend in hydro-climatic data series as the lower sensitivity to any sudden change 83 , 84 . To perform this test, it is essential to evaluate the presence of serial correlation within the data series 85 . A positive serial correlation can support the expected number of bogus positive products in the MK test 86 . For this reason, the serial correlation must be excluded prior to applying the MK test. To eliminate the serial correlation, the trend free pre-whitening (TFPW) technique proposed by Yue and Wang 85 was used.

The MK test was calculated using Eq.  1 :

In a time series, K i , i  =  1, 2, 3, ……….n , the value of S is supposed to be similar as the normal distribution with a mean 0 and while the discrepancy of statistics S has been computed using Eq.  2 :

The Z MK value is used to find out that the time series information is demonstrating a significant trend or not. The Z MK value is computed using Eq.  3 :

The positive and negative values of Z in a normalized test statistic reflect the increasing and decreasing trend, respectively, while the Z having 0 values reflects a normal distributed data series.

Method for change point detection

In the present study, we utilized Pettitt test proposed by Pettitt 37 and Standard Normal Homogeneity test (SNHT) developed by Alexandersson and Moberg 35 to explore the presence of the abruptly shifting change points in the time series annual and seasonal rainfall dataset for all meteorological sub-divisions.

Pettitt Test

The Pettitt test is a distribution-free rank based test, used to discover noteworthy changes in the mean of the time series. It is more helpful when the hypothesis testing about location of a change point is not necessary. This test has been used extensively to identify the changes observed in climatic and hydrological data series 87 , 88 . When length of a time series is represented by t and the shift take place at m years, the consequential test statistics are expressed as given in Eq. ( 4 ).The statistic is similar to the Mann- Whitney statistic, which characterized by two samples, such as k1, k2…, km and km+1, k2…, kn:

where \(\mathrm{sgn}\) in Eq.  4 is defined by Eq.  5 :

The test statistic Ut,m is calculated from all haphazard variables from 1 to n . The majority of distinctive change points are recognized at the point where the magnitude of the test statistic | Ut,m  | is highest (Eq.  6 ).

The probability of shifting year is estimated when | Ut,m  | is maximum following Eq.  7 :

If the p- value is less than the significance level α, the null hypothesis is considered to be rejected.

Standard Normal Homogeneity Test (SNHT)

The standard normal homogeneity test is also known as the Alexanderson test. This test is applied to detect sudden shift or presence of change point in time series of climatic and hydrologic datasets. The change point has been detected following Eq.  8 :

The change point refers to the point, when T s attains maximum value in the data series. The T m is derived using Eq.  9 :

where \(\bar{m}\) represents the mean and s represents the standard deviation of the sample data.

Buishand Rang Test

The Buishand range test is also called as Cumulative Deviation test, which is calculated based on the adjusted biased sums or cumulative deviation from mean. The change point using this test is detected following Eqs.  11 and 12 :

The \(S/\sqrt{n}\) is then estimated using the critical values proposed by Buishand 38 .

Methods for innovative trend analysis

The innovative trend analysis, proposed by Şen 55 , was applied to detect the trend in long-term time series rainfall data of India. The innovative trend analysis has greatest advantages over the MK test and other parametric and non-parametric statistical tests, which is that it does not need any assumptions like non-linearity, serial correlation and sample numbers. However, the graph of this test is being plotted on a Cartesian coordinate system depending on a sub-section time series. As per the method’s requirement, the time series rainfall data has been sub-divided in two time series data sets: 1901–1957 and 1958–2015. Both segments were fixed in an ascending order. Typically the first sub-series (x i ) was represented on x axis, while the other sub-series (y i ) was represented on y axis. The data is plotted on the 1: 1 line. If the data is plotted along the 1:1 line, it indicates simply no trend within the time series. If the data is being plotted above the 1:1 line, it will state that the moment series shows an increasing trend, and if the data falls under the 1:1 line, it will indicate the negative trend 56 , 58 . If the scatter plot is more close to the 1: 1 path, the trend alter slope of that time period series will be weaker, along with the farther far from the 1:1 tier it is, the particular stronger the excitement change incline in the time series 55 . Often the straight-line trend slope displayed by the ITA method can be expressed as by the Eq.  13 56 :

Where s is the indicator of trend having the positive values that represent the increasing trend, while the negative values represent the decreasing trend. The \(\bar{{\rm{y}}}\) and \(\bar{{\rm{x}}}\) are the arithmetic average of the first sub-series (xi) and second sub-series (yi) respectively. Furthermore, n is represented by the number of collected data products. The indicator is then multiplied by 10 for comparing with MK test 61 .

Method for analyzing rainfall changes

To calculate atmospheric oscillations on rainfall trend variation, winter and summer precipitations and moisture divergence during 1979–2015 on 1.25° × 1.25° grids were obtained from the European Centre for Medium Range Weather Forecasts (ECMWF), ERA-5 ( http://apps.ecmwf.int/ datasets/data/interim-full-daily). The ERA5-Interim is the most recent ocean-atmospheric changes reanalysis datasets available since 1979 forwards. In addition, low cloud cover dataset was also derived from the ECMWF ERA5 data to assess the effect of cloud cover on rainfall variation 89 . We have quantified the influence of atmospheric circulation changes on the trend patterns in rainfall. At first, we have detected a recent significant change point of annual mean rainfall based on Pettit test for the period 1979–2015 in India and observed that the mean rainfall has a change point after 2000. Then, the change in circulation of the two periods before and afterword the changes are quantified by subtracting 1979–2000 from 2001 to 2015using the ECMWF ERA5 reanalysis data. The GrAdS software ( http://cola.gmu.edu/grads/ ) was used to prepare the spatial maps.

Method for future forecasting

Several probabilistic and deterministic methods like ARMA, ARIMA, SARIMA are usually employed to predict the hydrological and climatic datasets. These techniques have many drawbacks like serial correlation, non-linearity, and biasness to predict the non-linear hydro-climatic data sets. Therefore, the newly developed artificial intelligence (AI) models can able to overcome this drawback. Hence, the application of AI models is now widely popular to solve the environmental problems. However, in the present study, we employed artificial neural network, a popular AI model, to predict and forecast the rainfall of thirty-four meteorological sub-divisions. The ANN is a non-linear black-box AI model. This model works like parallel-distributed information processing system which reflecting biological structure of brain as it comprised of simple neurons and links that process information to establish an association between the inputs and outputs. Likewise the function of brain, the ANN model is working using feed-forward multilayer perceptron algorithm. This structure is consisting of three layer, such as an input layer, one or more hidden layer and an output layer. We selected the number of neurons in the hidden layer by trial and error procedure. The ANN learning is based on the structure and functions of biological neural networks i.e. enclosing adjustment to the synaptic links that present in the core of the neurons. The multi-layer perception (MLP) is one of the most extensively employed neural network typologies. The MLP with two hidden layers is the most used classifier for the stationary pattern classification. MLPs are generally functioned using the backpropagation algorithm. The back-propagation principle flourishes the errors from the network and permits adjustment of the unseen units. Two crucial characteristics of multilayer perceptron are:

(1) The non-linear processing elements (PEs) having non-linearity that should be smooth, so that the logistic functions and hyperbolic tangent can be frequently employed; and

(2) Their enormous interconnectivity (any constituent of a given layer feeds all the constituents of the next layer).

In the present study, we divided the whole rainfall datasets into training and testing datasets. Then we normalized the data and applied ANN model for predicting the annual rainfall. We applied ANN model again and again on the same rainfall datasets by changing the model’s parameters like seed, momentum, learning rate until the best ANN model achieved for prediction rainfall. In addition, we evaluated the performance of ANN for predicting using Root Mean Square Error (RMSE) techniques. When we achieved the best ANN model, we fixed the model parameters and applied on the rainfall data to forecast rainfall up to 2030. However, the rainfall prediction and forecasting were performed in Weka software (version 3.9) ( https://weka.informer.com/3.9/ ) and the mappings were done in Arc GIS (version 10.3) software.

Method for change rate calculation in rainfall time series data

The simple statistical method, percentage change is applied to calculate the change rate of the annual and seasonal rainfall for pre change and post change point. This method is very simple but the function of this method is much effective. It is calculated using Eq.  14 .

Spatial mapping using Kriging interpolation method

In this study, the ordinary kriging method was employed to interpolate the statistical data and prepare the spatial rainfall map. The ordinary kriging method works on the data having statistical properties or spatial autocorrelation and this method employed the semi-variogram model to represent the spatial linkage (autocorrelation). The semi-variogram model evaluates the power of spatial association as a function of distance between the data. Kriging is used to estimate the values Z*( x 0 ) at the point x 0 expressed 90 in Eq.  15 and the estimation of error variance \({\sigma }_{k}^{2}({x}_{0})\) , expressed in Eq.  16 .

Where λ i refers to weights; μ refers to LaGrange constant; and ( x 0  −  x i ) represents the semi-variogram value equivalent to the distance between x 0 and x i 91 , 92 . Nielsen and Wendroth 93 suggested that a semi-variogram is comprised of the regionalized variable theory and intrinsic hypotheses and it is expressed as follows:

Where, γ ( h ) represents semi-variance, h represent lag distance, Z refers to the rainfall-related parameters, N(h) is the number of couples of locations divided by the lag distance h, \(Z({x}_{i})\) , and \(Z({x}_{i}+h)\) refers to the values of Z at the positions x i and x i  +  h 94 . The empirical model of semi-variogram generated from collected data which was fitted to the theoretical semi-variogram. It was utilized to create geo-statistical properties which further include nugget structured and the sill variance as well as the distance parameter. In addition, the nugget-sill ratio has been calculated in order to characterize the spatial dependency of the statistical values. A nugget-sill ratio of 75% reflects a weak spatial dependency; or else the dependency is considered moderate.

Furthermore, all mappings in the present work were performed using kriging interpolation method in Arc GIS (version 10.3) software.

Results and Discussion

Descriptive analysis of annual rainfall.

We calculated the descriptive statistics of the annual rainfall since 1901 to 2015 for thirty-four meteorological sub-divisions of India. Results show that the South Indian meteorological divisions i.e. Kerala, Tamil Nadu, and Konkan & Goa have observed the highest average rainfall (3396.64 mm. 2930 mm. and 2974 mm. respectively). While, the minimum average rainfall has been recorded in the sub-divisions of West India meteorological divisions i.e. West Rajasthan (288.74 mm.), Saurashtra and Kutch (494.27 mm.), Haryana (535.47 mm.), Delhi and Chandigarh (596.16 mm.). The standard deviation of rainfall for whole India varies from 1242.04 to 108.99 mm. The highest variation (standard deviation) in rainfall was observed in Arunachal Pradesh meteorological sub-division, followed by Coastal Karnataka (480.98 mm.), Konkan & Goa (478. 49 mm.), while the minimum variation was recorded in Western Rajasthan (108. 99 mm., followed by North Interior Karnataka (135.33 mm), Haryana Delhi and Chandigarh (142. 64 mm). The skewness of the rainfall ranges from −0.81 to 1.05 for all sub-divisions of India. The negative skewness was found in Tamil Nadu (−0.81), Bihar (−0.39), Konkan & Goa (−0.21), Jharkhand (−0.15), West Uttar Pradesh (−0.09), East Madhya Pradesh (−0.02) and Madhya Maharashtra (−0.001). While rest of the meteorological sub-divisions were observed positive skewness.

The arithmetic mean is not significant (robust) to local variations 95 . Therefore, LOWESS regression curve was applied on the seasonal rainfall to minimize the local variation. A cluster of researchers used this statistical method and achieved satisfactory findings over the arithmetic mean 96 , 97 . The findings of LOWESS curve indicate an increasing pattern of annual rainfall upto1965, while a decreasing pattern of rainfall was found after the year 1970. (Fig.  1b ). The findings of LOWESS curve for the winter season (Fig.  1c ) showed that the increasing rainfall pattern was observed for the periods of 1935–1955 and 1980–1998. A sudden decrease in the trend was observed after 1955 and 1998. In the case of summer and monsoon season (Fig.  1d,e ), the curve indicates that the negative trend was observed after the 1960. The identical result was found in the work of 98 . Whilst, in the case of post-monsoon (Fig.  1f ), the LOWESS regression curve indicates the positive trend was found during 1925–1935 and 1955–1965. The negative trend for post monsoon was observed after 1995.

Long Term Pattern and variation of average annual and seasonal rainfall

We used the coefficient of variation techniques to explore the rainfall variation for all meteorological sub-divisions. The Fig.  2a–e showed the spatial mapping of variations of average annual and seasonal rainfall over India using ordinary kriging interpolation method which is geo-statistical approach. The findings of spatial mapping of rainfall variation showed that the meteorological sub-divisions of Western India were recorded highest rainfall fluctuations. The minimum rainfall fluctuation was registered in Assam and Meghalaya (11.35%) metrological divisions, followed by Sub-Himalayan meteorological divisions like West Bengal (12.25%), Orissa (12.92%) and extreme South Indian divisions like Kerala (14.16%), Coastal Karnataka (14.42%) and North Interior Karnataka (14.67%). This result indicates that these states have been experiencing very less inconsistent rainfall trend for 115 years. While, the highest rainfall variation was found in Saurashtra and Kutch (41.14%), followed by Western Rajasthan (37.75%), Arunachal Pradesh (33.23%) and Gujarat region (30.46%) indicating the irregular occurrences of rainfall throughout the year.

Meteorological subdivision wise spatial variations using the coefficient of variation (CV) in annual and seasonal rainfall where figure ( a ) shows annual, ( b ) winter, ( c ) summer, ( d ) monsoon, and ( e ) post monsoon rainfall pattern.

The analysis of distribution and fluctuation of rainfall over Indian meteorological divisions’ show that the occurrence of winter season rainfall was comparatively less than the other seasons during 1901–2015. The maximum variation of rainfall was reported in Konkan & Goa (253.37%), followed by Coastal Karnataka (187.70%), Gujarat region (185.12%), Saurashtra & Kutch (174.88%) and Madhya Maharashtra (162.06%) indicating the unstable incidence of rainfall in these meteorological units. More consistent rainfall with low variations was observed in Himachal Pradesh (41.89%), followed by Arunachal Pradesh (43.15%), Uttarakhand (48.69%) and Assam & Meghalaya (52.21%) meteorological units (Fig.  2c ). Result show that, the rainfall in summer season had not recorded a significant variation, whilst the maximum variations of rainfall was observed in the meteorological divisions of Western India. The lowest rainfall variation was found in the meteorological divisions of Northeastern India and extreme South India.

The study of diverge distribution of rainfall over India for monsoon season indicates that the maximum rainfall was registered in Saurashtra & Kutch (42.67% and 157.64%), followed by West Rajasthan (39.93% and 145.35%), Arunachal Pradesh. Whereas, the highest rain fall variation in post-monsoon season was found in Punjab (36.18% and 138.65%), followed by Gujarat region (31.38% and 127.66%) indicating unstable rainfall incidences (Fig.  2d,e ). The lowest rainfall fluctuation of monsoon season was recorded in the meteorological divisions of Assam & Meghalaya (12.69%), followed by Orissa (13.34%), Sub Himalayan West Bengal (13.94%) and Coastal Karnataka (15.29%). While, in the post-monsoon season, the lowest rainfall fluctuation was observed in Kerala (26.21%), followed by Tamil Nadu (29.46%) and South Interior Karnataka (36.44%).

Meteorological sub-division wise trends of Annual and Seasonal Rainfall

We calculated the annual and seasonal rainfall trend for thirty-four meteorological sub-divisions using non-parametric Mann-Kendall test (Table  1 ). The Table  1 shows the five shades of brown color based on the intensity of z value of Mann-Kendall test at 0.05 significance level which indicates that darker the shade of brown color, higher the negative z value and vice-versa. Results show that five sub divisions for annual rainfall were found in very dark shade zones (Nagaland Mizoram Manipur & Tripura, East Madhya Pradesh, Jharkhand, East Uttar Pradesh, Chhattisgarh and Kerala) which suggests that these sub-divisions were experienced highly negative trend of rainfall (more than −2 of z value). The z value of MK test ranges from 0 to −2 which could be found in eight meteorological sub-divisions, i.e. Bihar, Orissa, Assam & Meghalaya, West Uttar Pradesh, Uttarakhand, Himachal Pradesh, Vidarbha and East Rajasthan (dark brown color in Table  1 ). The z value having the positive trend varies from 0 to 2 which were observed in the eleven meteorological sub-divisions that implies the increasing nature of rainfall over these divisions, while the six meteorological sub-divisions recorded highly positive trend of more than 2 of z value (very light shade of brown color in Table  1 ), which indicates the significant increase of rainfall over time in these meteorological units.

The two meteorological division (Madhya Maharashtra and North Interior Karnataka) for summer season and seven meteorological divisions (Assam & Meghalaya, Orissa, Bihar, Punjab, East Rajasthan, Chhattisgarh and Kerala) for monsoon season were recorded highly negative trend in rainfall having >2 of z value (very dark shade rows in Table  1 ). On the other hand, no meteorological divisions for winter and post-monsoon season were recorded as highly negative trend (more than >2 of z value) in rainfall, while most of the sub-divisions were detected as negative trend having the z value 0 f 0 to −2 (dark shade rows in Table  1 ).

The Change Point Detection analysis for annual and seasonal rainfall

The above-mentioned analysis (Table  1 ) explained that few meteorological sub-divisions were detected as significant negative trend having the z value of >−2. However, several researchers claimed that the actual trend could not be detected if we apply the MK test on overall hydro-climatic datasets. Therefore, they highly recommended to apply the change detection techniques before the application of MK test. Hence, we utilized change detection methods like Pettitt test, SNHT test and Buishand range test for detecting the abrupt change point in the rainfall datasets in thirty-four meteorological sub-divisions (Supplementary Table  1 ). The Supplementary Table  1 shows that the annual and seasonal rainfall of all meteorological sub-divisions had the abrupt change point which were detected by mentioned three change detection techniques. Furthermore, we selected the change point for annual and seasonal rainfall in each sub-divisions based on the performances (p value) of these tests (Supplementary Table  1 ). Therefore, these abrupt change points suggest that the rainfall datasets had no monotonous trend. The selected change point for seven meteorological sub-divisions (East Uttar Pradesh, West Uttar Pradesh, Punjab, and Gujarat region, Saurastra & Kutch, Coastal Andhra Pradesh and Tamil Nadu) were after 1990s. The nineteen meteorological sub-divisions had the abrupt change point during the period of 1950–1980. The abrupt change point year for the rest of the meteorological divisions were detected before 1940.

Change point wise annual and seasonal variation analysis

We computed the annual and seasonal rainfall variation in pre and post change point wise for all meteorological sub-divisions to explore the dynamics of the intensity of annual and seasonal rainfall variations after the change point. Therefore, we can consider this change point wise rainfall variation analysis as a validating method for the relevancy of uses of the change point methods and further research. Hence, to prove this statement, we computed and prepare the spatial map of annual and seasonal rainfall variation for pre change point (Fig.  3a–e ) and post change point (Fig.  3f–j ) using coefficient of variation method. Results show that highest fluctuation (25% to 239.02%) in annual and seasonal rainfall was observed in the sub-divisions of Western India and North-Western India (3a-e). Whereas, the minimum fluctuation in annual and seasonal rainfall having the CV of 0.36% to 7.76% was observed in the sub-divisions of North Eastern India, Eastern India and Northern India indicating the consistent rainfall occurrence in these region.

Spatial variation of rainfall measured using the coefficient of variation for pre-change point where figure ( a ) shows annual rainfall, ( b ) winter, ( c ) summer, ( d ) monsoon, and ( e ) post monsoon; post-change point where figure ( f ) shows annual rainfall, ( g ) winter ( h ) summer ( i ) monsoon and ( j ) post monsoon; spatial changes in the rate of rainfall where figure ( k ) shows annual rainfall, ( l ) winter ( m ) summer ( n ) monsoon and ( o ) post monsoon.

The findings of annual and seasonal rainfall analysis in post change point phase show that the large areas of the country like the sub-divisions of Western India, Central India, and South Western India observed high fluctuation having the CV of 22% to 209.82%. In case of monsoon and post monsoon season, the sub-divisions of North Western were experienced by high variation of rainfall suggesting the high tendency of inconsistently rainfall occurrences in these regions. While the lowest rainfall variation having the CV of 0.75% to 11.64% was recorded in sub-divisions of North-Eastern India, North India, East India of winter and summer rainfall and Central India of Post monsoon rainfall. The intensity of minimum rainfall variation range was increased in post change point phase which suggests that inconsistency rainfall events were observed. However, in the post change point phase, the area coverage of lower variation of rainfall incidences were reduced significantly that implies the climate change and validation of the application of change point detection methods.

Rainfall change rate analysis

In the present study, we computed the rainfall change rate for annual and seasonal rainfall in all meteorological sub-divisions based on the calculation between the rainfall data of pre and post change point phase. The Fig.  3k–o shows the spatial mapping of rainfall change rate for annual and seasonal rainfall. Results show that sub-divisions of Western India were observed highest change rate (47% to 58.54%) in annual rainfall, while the sub-divisions of whole country except West India were registered the highest change rate (−0.5% to −1.77%) in winter rainfall. In case of summer and monsoonal rainfall, the sub-divisions of western were observed highest negative change rate (34.22% to 67.04), while the South India and North East India were recorded highest negative change rate in post monsoon rainfall. Furthermore, the North India and Central India were recorded minimum change rate, whereas, the Western India of winter and post-monsoon rainfall. This analysis signifies that the amount of rainfall occurrences was decreased significantly after change point.

Change point wise seasonal rainfall trend analysis

We applied MK test on the datasets of pre and post change point of seasonal rainfall in each of the meteorological sub-divisions as per the recommendation of many scientists. The Table  2 reported the seasonal rainfall trend for pre change point in all sub-divisions Results show that eight meteorological sub-divisions in monsoon, six sub-divisions in post monsoon, fourteen sub-divisions in both summer and winter season were recorded negative trend having the z value of 0 to −2 (dark shade of brown color rows in Table  2 ). While, three sub-divisions in summer and four sub-divisions in winter seasons were recorded highly negative trend having z value of > −2 (very dark shade of brown color rows in Table  2 ). Rests of the meteorological sub-divisions were detected as positive trend (lighter shade of brown color in Table  2 ) except one division (Sub Himalayan West Bengal & Sikkim) which detected has no trend. From the analysis, it can be stated that non-monsoonal seasons were recorded as declining trend.

The Table  3 showed the trend analysis using MK test for post-change point seasonal rainfall. Seventeen meteorological sub-divisions in monsoon, sixteen sub-divisions in post monsoon, seventeen sub-divisions in summer and twenty one sub-divisions in winter seasons were observed the negative trends having the z value of 0 to −2 indicating that huge amount area was recorded declining rainfall than the pre-change point phase (dark shade of brown color rows in Table  3 ). The increasing of meteorological divisions which have the negative trend from the pre change point (Table  3 ). This circumstance implies that the declining trend of rainfall was increased manifold after post change point which shows the sign of climate change. On the other hand, rest of the meteorological sub-divisions were experienced the insignificant positive trend having the z value of <0.5.

Innovative trend analysis for seasonal rainfall

Several researchers suggested that non-parametric statistical techniques like mann-kendall test has many drawbacks like the presence of serial correlation within the data sets, non-linearity and most importantly sample size which could have ability to influence the result. Therefore, Sen 55 developed innovative trend method which can overcome the mentioned drawbacks, especially the problem sample size. Sen 55 reported that innovative trend can effectively able to detect the trend on any numbers of sample size and presence of serial correlation. Hence, we used innovative trend to calculate the trend for seasonal rainfall (winter, summer, monsoon and post monsoon) in thirty-four meteorological sub-divisions (Supplementary Figure  1 – 4 ). We computed D value of innovative trend to compare the intensity of trend achieved by MK test. The spatial mapping using D values of innovative trend for seasonal rainfall was presented in Fig.  4a–d . However, the Supplementary Table  2 showed the slope value of innovative trend for seasonal rainfall in all meteorological sub-divisions. The findings indicate that the negative trend was detected in the sub-divisions of North Eastern, Central and Southern India for summer season. The results were quite identical with the findings of MK test, but the magnitudes of the trend were different which stated that the region was experienced strong negative trend (Fig.  4 ). Although the highly positive trend was detected in the sub-divisions of Rajasthan part and Jammu-Kashmir region which does not imply that the rainfall occurrences was increased, but the regions were received more or less consistent amount of rainfall throughout the time periods, while little amount of rainfall was increased in few years over these regions which is the reason for positive trend. However, in case of monsoon rainfall, the negative slope of innovative trend was detected in the sub-divisions of North Eastern part, Eastern part and some parts of the central India, whereas, rest of the sub-divisions were detected as insignificant positive slope of trend. The sub-divisions of North Eastern states, Bihar, Orissa, Jharkhand, Western Ghat and Punjab regions were experienced very strong negative slope of innovative trend, on the other hand, rest of the part were recorded insignificant positive slope of trend for post monsoon rainfall. In case of winter rainfall, the meteorological sub-divisions of Central part of India, Southern India, Western Ghat regions were observed the negative trend, while the sub-divisions of North Eastern, Western and Eastern part of India were experienced positive slope of trend. Therefore, the findings of MK test and innovative trend analysis were highly identical, however, few states where no significant trend were detected using MK test, but in the case of innovative trend, those region were come under negative trend. However, the findings from both trend detected methods clearly stated that India has been experiencing fewer downpours than the expected rainfall since last 30 years that clearly points out about the climate change. Several researches established that innovative trend can able to detect trend effectively over the others non-parametric test 18 , 99 , 100 , . Therefore, we considered innovative trend as a tool of intensity of trend measurement over the other techniques and results show that the highly negative trend was detected in most of the sub-divisions indicate about the climate change.

The spatial variation of slope of innovative trend for ( a ) summer, ( b ) monsoon, ( c ) post monsoon and ( d ) winter.

Micro level rainfall change rate analysis

In the present study, we attempted to analyze the change rate of annual rainfall for each and every year in thirty-four meteorological sub-divisions. We computed the change rate by calculating the departure of year wise average rainfall from the long-term average rainfall. The heat map was used to show the dynamics of year wise rainfall change rate for all sub-divisions (Fig.  5 ). This heat map was generated from R software (version 3.5.3) ( https://cran.r-project.org/bin/windows/base/old/3.5.3/ ) using the ggplot2 package 101 ( https://cran.r-project.org/web/packages/ggplot2/index.html ). The intensity of change rate was represented by the shades of red and green color indicating the highly negative change rate and vice versa. Results show that the after 1970, all meteorological sub-divisions were observed the negative departure in almost all years from long-term average rainfall by 50 mm.−2000 mm. (Fig.  5 ). While the sub-divisions of North-Eastern India and North India were recorded positive change rate only for few years because of the occurrences of excessive rainfall that was happened occasionally by monsoon burst, ELSO effect (Kripalini et al . 2003) and local climatic effect. However, the highest negative change rate was observed in the sub-divisions like Arunachal Pradesh, Nagaland Manipur, Mizoram & Tripura, Kerala, Western Uttar Pradesh, Rajasthan, Uttarakhand and Himachal Pradesh by more than −2000 mm. While the positive change rate was mainly detected before change detection year or 1970 in all sub-divisions. The positive change rate was varied from 0–2000 mm. However, few sub-divisions like Kerala, Nagaland Manipur Mizoram & Tripura, and Coastal Karnataka.

Heatmap represents the departure of rainfall from normal rainfall for all meteorological subdivision. (N.B. CK- Coastal Karnataka, K & G – Konkan and Goa, HP- Himachal Pradesh, UK- Uttarakhand, EMP- Eastern Madhya Pradesh, CG- Chhattisgarh, JK – Jharkhand, J & K – Jammu and Kashmir, S & K- Saurashtra and Kachcha, GR – Gujarat region, GWB – Gangetic West Bengal, NIK – North Interior Karnataka, SIK – South Interior Karnataka, TN – Tamilnadu, CAP – Coastal Andhra Pradesh, WUP – western Uttarpradesh, EUP – Eastern Uttarpradesh, ER – Eastern Rajasthan, WR – Western Rajasthan, HD & C- Haryana Delhi and Chandigarh, WMP – WesternMadhyapradesh, MM – MadhyaMadhyapradesh, A & M- Assam and Meghalaya, SHW – Sub Himalaya West Bengal, NMM & T- Nagaland, Manipur, Mizoram and Tripura, AP- Arunachal Pradesh.

Rainfall prediction and forecasting

The above mentioned analyses show that most of the meteorological sub-divisions of India were experienced significant decrease in rainfall and this decrease was become stronger after the change point. Therefore, the situation of rainfall incidence became critical post change point. Therefore, if the present situation of rainfall occurrences continues with the same intensity, the intensity of rainfall occurrences and their distribution in future will be more worsen. Therefore, the prediction and forecasting of rainfall is become essential for water resource management and planning. Several soft computing machine learning techniques are available for predicting and forecasting rainfall, but MLP based artificial neural network has become widely popular among the researchers as it is easy to compute and generate high quality product 102 , 103 . In this study, we applied MLP-ANN on the annual rainfall data for prediction. We set the parameters of artificial neural network models again and again until the best prediction had not achieved which we evaluated using RMSE techniques. We fixed the model’s parameters and applied on the rainfall of thirty-four meteorological sub-divisions for prediction. In some cases, we changed some of models parameters for predicting in rainfall. However, the Supplementary Table  3 shows the plot between the best MLP-ANN generated model and observed rainfall for thirty four meteorological sub-divisions that indicates that there is close adjacency between predicted and observed rainfall. The error measures like RMSE and MAE (in mm.) were used to quantify the closeness between predicted and observed rainfall. The Supplementary Table  4 reports the 15 years rainfall forecasting up to 2030 for all meteorological sub-divisions. We selected the rainfall of 2020 and 2030 as the representative for spatial mapping (Fig.  6 ) (for the rainfall forecasting data of all years was presented in Supplementary Table  4 ). The findings of spatial mapping of future forecasting show the more declining amount of rainfall.

Spatial rainfall forecast for 2020 ( a ) and 2030 ( b ).

In 2020, the maximum rainfall incidence will be observed in the sub-divisions of Coastal Karnataka, Konkan & Goa, Kerala (>2500 mm rainfall) and the minimum rainfall event will be placed on the West Rajasthan, Tamil Nadu, East Rajasthan, Rayalseema (<400 mm rainfall) (Fig.  6a ). On the other hand, in 2030, the highest rainfall will be occurred in the sub-divisions of Coastal Karnataka, Konkan & Goa, Kerala (>2000 mm rainfall), while the lowest rainfall event will be found in the sub-divisions like West Rajasthan, East Rajasthan, Tamilnadu, Rayalseema (<300 mm rainfall) (Fig.  6b ). The findings of future forecasting analysis point out that the occurrences of rainfall will be decreasing gradually for some meteorological stations, while the rainfall will be increased in some meteorological sub-division. The places of rainfall occurrences over India will be remained as like identical pattern of present and earlier rainfall incidences. The findings also show that the extreme rainfall events will be increased in future that will be lead to flooding situation.

Causes of rainfall variation

India encompasses large areas with moderate to high convective precipitation, while low convective rainfall rate occurred which was brought excessive moisture from the Indian Ocean to the in land areas, which is unfavorable for the formation of rainfall (Fig.  7a–f ). This caused a decreased in mean rainfall during monsoon season, which has been distributed in the extreme southern and mid-central divisions of the country (Fig.  7 ). The northeasterly wind was strengthened in the whole country, which reduced the invasions of cooler air, led to decline in rainfall all over India in the winter season. By contrast, most of the divisions had moderate to high mean convective precipitation rate, which increased rainfall to some extent. The low cloud has increased all over the India, and a few cloud covers will enhance the consolidation impact of atmosphere on the solar radiation, and hence it leads to declining rainfall trend. Moreover, high mean total precipitation rate has been influencing across the country except for eastern division, which triggered uneven downdraft pattern and leading to more clear sky days during the recent study period (1979–2017). Most of the regions in India exhibited a declining vertically integrated moisture divergence which triggered by a significant decreasing rainfall changes.

Spatial variations of differences in ( a ) convective precipitation in monsoon season, ( b ) convective rainfall rate in winter season, ( c ) low cloud cover, ( d ) mean convective precipitation rate, ( e ) mean total precipitation rate, and ( f ) mean vertically integrated moisture divergence between the recent period of 2001–2015 and 1979–2000.

Marumbwa et al . 104 suggested that the analysis of historical rainfall trend is very crucial in several fields like water resource management, sustainable agricultural planning, ecosystem management and health sector. The rainfall data of 115 years were used to investigate the variability of annual and seasonal rainfall in very detail and analyzed the trend in several ways for thirty-four meteorological sub-divisions. The findings of 115 years annual and seasonal rainfall variation analysis show that the highest rainfall variation was found in the sub-divisions of Western India of annual and summer, Western India and South Western India of winter, North India and Western India of monsoon and Western, North Western and some parts of Eastern India of post monsoon rainfall. While the lowest variation was found in the sub-divisions of North Eastern and Eastern India. The identical result was reported by several previous studies 32 , 45 , 66 , 71 , 105 . The findings of MK test on annual and seasonal rainfall report that thirteen meteorological sub-divisions were observed negative trend, while rest of the sub-divisions were recorded positive trend. Rajeevan et al . 106 and Guhathakurta and Rajeevan 46 reported that Jharkhand, Chhattisgarh and Kerala were observed negative trend, while eight sub-divisions like Gangetic West Bengal, Coastal Andhra Pradesh, Kanakan & Goa, North Interior Karnataka, Rayalessema, Jammu &Kashmir were recorded decreasing trend over time. These results are totally identical with the findings of the MK test of the present study (Very dark shade of brown color rows in Table  1 ). Mirza et al . 107 reported that the annual and seasonal rainfall was more or less consistent in sub-divisions sub-himalayan Bengal and Gangetic Bengal which was by and large similar with the findings of MK test in the present study (Table  1 ).

Mondal et al . 32 used Pettitt test and SNHT test for detecting change point in the annual rainfall for all meteorological sub-divisions for their study and reported that twenty-one sub-divisions had the change point using Pettitt test between 1950–1966. While we detected eighteen sub-divisions that have change point between 1950–1966 (Supplementary Table  1 ). Goyal 98 documented that the most probable change point was 1959, while in the present study, most of the change points were detected in between 1950–1966 which can conclude that the findings of present work are identical with the work of Goyal. Therefore this study concludes that 115 years annual rainfall had not the monotonous trend and it needs for further research to detect the trend and its magnitude accurately. Based on which planners and scientists can propose the developmental and management plan.

Furthermore, MK test was employed on the annual and seasonal rainfall data of both pre and post change point phase to detect the exact and accurate trend as several scientists recommended to apply MK test based on the analysis of change point to achieve the accurate trend 34 , 38 , 108 . The misleading results could be found if the MK test applies first 38 . However, The findings of MK test on the annual and seasonal rainfall for pre change point phase stated that among the 34 meteorological divisions, 8 sub-divisions of each monsoon and post monsoon, 17 sub-divisions of summer and 18 sub-divisions of winter season were detected insignificant negative but trend (Table  2 ). Parthasarathy and Dhar 109 documented that most of the sub-divisions were recorded positive trend for the period 1901–1970 that was the change point year for the most of the sub-divisions in the present study (Table  2 and Supplementary Table  1 ).

The findings of the MK test for post change point phase showed that the negative significant trend was detected in the 17 meteorological sub-divisions of monsoon, 15 sub-divisions of post monsoon, 19 sub-divisions of summer and 21 sub-divisions of winter season (Table  3 ). Jain and Kumar 110 reported that 4 river basins of India were detected as increasing trend, while 13 river basins were observed significant negative trend in monsoon rainfall. In case of annual rainfall, 15 river basins were recorded negative trend. Therefore, we can state that these findings are supporting the present work. Guhathakurta et al . 111 reported that significant negative trend was found in NW and Central and Peninsular India for the period of 1951–2011. Several studies also reported that the significant amount of rainfall was decreased over time, especially after 1950 44 , 46 , 81 , 105 , 112 .

To the best of authors’ knowledge, the application of recently developed innovative trend analysis to detect rainfall trend was very rare. We found the work of Machiwal et al . 113 who applied innovative trend analysis to detect trend in rainfall and temperature of Indian arid region. Machiwal et al . 113 documented that increasing trend for monsoon rainfall was found in the arid region and this result is identical with the findings of the present study. However, the findings of Innovative trend analysis on annual and seasonal rainfall showed that negative trend was observed in the sub-divisions of NE, E, SE and S part of summer, NE and some parts of E India of monsoon, NE, E India of post-monsoon and W, SW, S India of winter rainfall. The identical finding was found in the work of Mondal et al . 32 . Taxak et al . 96 studied grid wise rainfall trend over India and documented that most of grids were received negative trend except seven grids which were observed positive trend. The findings of this study are totally identical with the present work.

Dash et al . 53 and Kumar et al . 11 documented that frequency of intense rainfall in many parts of Asia has increased, while the amount of rainfall and number of rainy days has decreased significantly. For this reason, the year wise departure study revealed that some of the years were observed positive departure from the long-term average rainfall by more than 1000 mm rainfall (Fig.  5 ). Even the identical findings can be found in the work of Goswami et al . 40 and they reported that the frequency and magnitude of extreme rainfall has amplified significantly, while the frequency and intensity of moderate rainfall has observed noteworthy decreasing trend.

In the present study, the change rate was maximum in the meteorological sub-divisions of Western and Central India in annual and monsoon rainfall, whereas the maximum change rate (%) for winter season was in the sub-divisions of Central, North East and Eastern India. Mondal et al . 32 reported that many parts of Western India and Central India were received very high negative change rate for monsoon rainfall, while Central and North Eastern parts of India were observed highly decreasing change rate. Furthermore, it can be concluding from the analysis that North Eastern part of India has been observed the worst effect of climate change than any other parts of India.

In the present study, the annual rainfall for thirty-four meteorological sub-divisions were predicted and forecasted up to 2030. In addition, the rainfall forecasting for 2030 showed an expected decline of about 5–10% in the overall rainfall of India (Fig.  6 ). However, numbers of studies were conducted to predict and forecast rainfall for India based on the average rainfall and they did not consider studying for all parts of the India. Guhathakurta 114 predicted the rainfall using neural network for Kerala. While Chakraborty et al . 49 predicted the south-west monsoon for India. They used average time series annual rainfall dataset of India. But in the present study, we considered rainfall datasets for thirty-four meteorological sub-divisions. Therefore, the findings of this study would not be generalized; rather it would be more accurate and can be act as the foundation of the developmental planning.

The temporary change in rainfall distribution significantly distresses the agricultural production. It will increase the drought protection and resilience plans under the changing climate conditions. Intergovernmental Panel on Climate change (IPCC) reported that upcoming changes in climate is to be likely to distress agriculture that will amplify the chance of hunger and water paucity, as well as the instructions on quicker thawing of glaciers 115 . Gosain et al . 116 pointed that the amount of freshwater in the river of India is probable to be diminish because of changing climate. This reduction, along with growing population might unfavourably affect a large population in India by 2050.

Large-scale ocean-atmospheric changes derived from the ECMWF ERA5 re-analysis data depicted that compared between 1979–2000 to 2001–2015 period, an increasing/decreasing convective precipitation rate, enhanced low cloud cover and inadequate moisture variance in the Indian ocean being transported to the northwest direction might have highly influenced the rainfall trend in India.Therefore, it is expected that this study will provide an insight for the management and development of agriculture and water resources in India to overcome the possible impacts of the climate change.

The basic and essential requirement for the management and planning of water resource, sustainable agricultural development and other sectors is the exploration of the spatiotemporal distribution and changing pattern of rainfall in any places. Hence, the present study investigated the variability and trend analysis of annual rainfall in several ways like overall data, change point wise (pre and post change point) using 115 years long-term annual and seasonal rainfall data of thirty-four meteorological sub-divisions. The present study shows that the overall annual and seasonal variability of rainfall was highest in the sub-divisions of Western India, while the lowest variability was found in Eastern and North India. The findings of MK test on overall annual and seasonal rainfall reports that the sub-divisions of North-East, South and Eastern India were detected significant negative trend, while the sub-divisions like Sub-Himalayan Bengal, Gangetic Bengal, Jammu & Kashmir, Konkan & Goa, Madhya Maharastra and Marathwada were recorded positive trend. Furthermore, the change detection techniques were utilized and selected the change point based on the performances of the techniques. The most probable change points were detected in between 1950–1966. Based on the change point year, the rainfall variability and trend analysis were again carried out for pre and post change point phase. The rainfall variability was increased significantly in most of the meteorological sub-divisions after post change point and similar kinds of findings were found when the rainfall trend was analyzed for post change point. To get better results of trend analysis, the innovative trend analysis was employed. The finding shows that most of the sub-divisions were recorded significant negative trend. Even some of the sub-divisions were detected as no trend using MK test, but the trend was detected using innovative trend analysis. However, the micro level change rate analysis was used in the present study and the results show that after 1960, most of the meteorological sub-divisions were recorded more than −500mm rainfall departure from the long-term average rainfall in many years, while few years were detected positive departure in few sub-divisions. Therefore, from the detail analysis, it is established that almost all of the sub-divisions were detected the negative trend and high variability after 1970. Even the year wise study also revealed that in which year and how much rainfall amount was departed. Hence, these detail information regarding historical data for whole country are very beneficial for the planning. One of the most striking features of developmental planning in the recent time is the forecast of the incoming event which can be found in any sector like finance, water resource and most importantly climatology. Therefore, in the present study, the rainfalls for all meteorological sub-divisions were forecasted using the advance AI models like artificial neural network. The findings of the rainfall forecasting show that 15% of rainfall will be declined in 2030 that indicates the alarming situations will be appeared for both environment and living world.

The economy of India is totally dependent on the rainfall either it is agriculture or industry. Therefore, the water resource is essential part of the progressive economy of India. But due to climate change, the world rainfall pattern has been disturbed. Therefore, many studies were carried out in the developed countries to quantify the pattern of the rainfall changes and formulate the management plan accordingly. But in case of India, very fewer studies were done to do so. The present study provides information in all aspects like rainfall variability and trend for overall and change point wise for annual and seasonal rainfall, rainfall change rate since change point year, year wise departure, and future rainfall and most importantly this study analyzes the causes of rainfall changes in India. Technically, the present study used several sophisticated techniques which have been admired worldwide by the scientists for providing high precision results. This type of studies has not been conducted for whole India. Therefore, the present study can be the full package and should be very much helpful to the Indian planners to proposing plans for small and large scale regions.

To formulate the management plan for the sustainable development of water resource based sectors and environment, the scientist of others countries can conduct the research like the present study as they need lots of information for developing plan regarding historical, present and future data which can be in any field like hydrology, climatology.

However, in the present study, we considered thirty-four meteorological sub-divisions for the research, but to be more accurate, micro level data like district wise data should be incorporated. Then the very high precision micro level management plan will be achieved. Even, the grid wise rainfall study using very advanced microwave remote sensing technology will be very useful for the planners. The ensemble machine learning techniques, deep learning techniques like long-short-term memory (LSTM) network can be used to achieve very high quality forecasting data.

Islam, T., Rico-Ramirez, M. A., Han, D. & Srivastava, P. K. A Joss–Waldvogel disdrometer derived rainfall estimation study by collocated tipping bucket and rapid response rain gauges. Atmospheric Science Letters 13 , 139–150 (2012).

ADS   Google Scholar  

Gajbhiye, S., Meshram, C., Singh, S. K., Srivastava, P. K. & Islam, T. Precipitation trend analysis of Sindh River basin, India, from 102‐year record (1901–2002). Atmospheric Science Letters 17 (1), 71–77 (2015).

Google Scholar  

Srivastava, P. K., Han, D., Rico-Ramirez, M. A. & Islam, T. Sensitivity and uncertainty analysis of mesoscale model downscaled hydro-meteorological variables for discharge prediction. Hydrological Processes 28 , 4419–4432 (2014).

Islam, T., Rico-Ramirez, M. A., Han, D., Srivastava, P. K. & Ishak, A. M. Performance evaluation of the TRMM precipitation estimation using ground-based radars from the GPM validation network. Journal of Atmospheric and Solar-Terrestrial Physics 77 , 194–208 (2012).

Srivastava, P. K., Mehta, A., Gupta, M., Singh, S. K. & Islam, T. Assessing impact of climate change on Mundra mangrove forest ecosystem, Gulf of Kutch, western coast of India: a synergistic evaluation using remote sensing. Theoretical and Applied Climatology 120 (3–4), 685–700 (2015).

Gupta, M., Srivastava, P. K., Islam, T. & Ishak, A. M. B. Evaluation of TRMM rainfall for soil moisture prediction in a subtropical climate. Environmental Earth Sciences 71 , 4421–4431 (2014).

Meshram, S. G., Singh, V. P. & Meshram, C. Long-term trend and variability of precipitation in Chhattisgarh State, India. Theoretical and Applied Climatology 129 (3–4), 729–744 (2017).

Bandyopadhyay, J., & Perveen, S. A scrutiny of the justifications for the proposed interlinking of rivers in India. Interlinking of rivers in India: Overview and Ken-Betwa link , 23 (2006).

Attri, S. D., & Tyagi, A. Climate profile of India. Environment Monitoring and Research Center, India Meteorology Department: New Delhi, India (2010).

Jain, A. A Study of Trends and Magnitude of Farmer Suicides in India’. International Journal of Advances and Scholarly Research in Allied Education 13 (2), 80–85 (2017).

Kumar, V., Jain, S. K. & Singh, Y. Analysis of long-term rainfall trends in India. Hydrological Sciences Journal–Journal des Sciences Hydrologiques 55 (4), 484–496 (2010).

Yang, P., Ren, G. & Yan, P. Evidence for a strong association of short-duration intense rainfall with urbanization in the Beijing urban area. Journal of Climate 30 (15), 5851–5870 (2017).

Chatterjee, S., Khan, A., Akbari, H. & Wang, Y. Monotonic trends in spatio-temporal distribution and concentration of monsoon precipitation (1901–2002), West Bengal, India. Atmospheric Research 182 , 54–75 (2016).

Tian, Y., Bai, X., Wang, S., Qin, L. & Li, Y. Spatial-temporal changes of vegetation cover in Guizhou Province, Southern China. Chinese Geographical Science 27 (1), 25–38 (2017).

Xia, J., She, D., Zhang, Y. & Du, H. Spatio-temporal trend and statistical distribution of extreme precipitation events in Huaihe River Basin during 1960–2009. J. Geogr. Sci. 22 , 195–208 (2012).

Rao, B. B., Chowdary, P. S., Sandeep, V. M., Rao, V. U. M. & Venkateswarlu, B. Rising minimum temperature trends over India in recent decades: implications for agricultural production. Glob Planet Chang 117 , 1–8 (2014).

Talaee, P. H. Iranian rainfall series analysis by means of nonparametric tests. Theoretical and applied climatology 116 (3–4), 597–607 (2014).

Sonali, P. & Kumar, D. N. Review of trend detection methods and their application to detect temperature changes in India. Journal of Hydrology 476 , 212–227 (2013).

Haan, C. T. Statistical methods in hydrology. Second edition. Iowa State University Press, Ames, Iowa 496pp (2002).

Piao, S. L. et al . The impacts of climate change on water resources and agriculture in China. Nature 467 (2), 43–51 (2010).

ADS   CAS   PubMed   Google Scholar  

Mann, H. B. Non-parametric tests against trend. Econometrica 13 , 245–259 (1945).

MathSciNet   MATH   Google Scholar  

Kendall, M. G. Time Series. Charles Griffin and Co. Ltd., London (1973).

Kendall, M. G. Rank correlation methods. Charles Griffin and Co. Ltd., London, U.K. (1975).

Sen, P. K. Estimates of the regression coefficient based on Kendall’s tau. J Am Stat Assoc 63 (324), 1379–1389 (1968).

Pingale, S. M., Khare, D., Jat, M. K. & Adamowski, J. Spatial and temporal trends of mean and extreme rainfall and temperature for the 33 urban centers of the arid and semi-arid state of Rajasthan, India. Atmos Res 138 , 73–90 (2014).

McGhee, J. W. Introductory statistics. West Publishing Co., New York (1985).

Batisani, N. & Yarnal, B. Rainfall variability and trends in semi-arid Botswana: implications for climate change adaptation policy. Applied Geography 30 (4), 483–489 (2010).

Tabari, H., Marofi, S., Aeini, A., Hosseinzadeh Talaee, P. & Mohammadi, K. Trend analysis of reference evapotranspiration in the western half of Iran. Agric For Meteorol 151 , 128–136 (2011).

Du, J. & Shi, C. Effects of climatic factors and human activities on runoff of the Weihe River in recent decades. Quaternary International 282 , 58–65 (2012).

Singh, P., Kumar, V., Thomas, T. & Arora, M. Changes in rainfall and relative humidity in different river basins in the northwest and central India. Hydrol Process 22 , 2982–2992 (2008).

Wang, S., Yan, M., Yan, Y., Shi, C. & He, L. Contributions of climate change and human activities to the changes in runoff increment in different sections of the Yellow River. Quaternary International 282 , 66–77 (2012).

Mondal, A., Khare, D. & Kundu, S. Spatial and temporal analysis of rainfall and temperature trend of India. Theoretical and applied climatology 122 (1–2), 143–158 (2015).

Zarenistanak, M., Dhorde, A. G. & Kripalani, R. H. Trend analysis and change point detection of annual and seasonal precipitation and temperature series over southwest Iran. Journal of earth system science 123 (2), 281–295 (2014).

Li, D., Xie, H. & Xiong, L. Temporal change analysis based on data characteristics and nonparametric test. Water Resour Manage 28 , 227–240 (2014).

Alexandersson, H. & Moberg, A. Homogenization of Swedish temperature data. Part I: Homogeneity test for linear trends. International Journal of Climatology: A Journal of the Royal Meteorological Society 17 (1), 25–34 (1997).

Buishand, T. A. Some methods for testing the homogeneity of rainfall records. Journal of hydrology 58 (1-2), 11–27 (1982).

Pettitt, A. N. A non‐parametric approach to the change‐point problem. Journal of the Royal Statistical Society: Series C (Applied Statistics) 28 (2), 126–135 (1979).

Villarini, G., Serinaldi, F., Smith, J. A. & Krajewski, W. F. On the stationarity of annual flood peaks in the continental United States during the 20th century. Water Resour Res 45 , W08417 (2009).

Sen Roy, S. & Balling, R. C. Jr Diurnal variations in summer season precipitation in India. International Journal of Climatology: A Journal of the Royal Meteorological Society 27 (7), 969–976 (2007).

Goswami, B. N., Venugopal, V., Sengupta, D. & Madhusoodanan, M. S. and xavier, P. K. Increasing Trend of Extreme Rain Events Over India in a Warming Environment. Science 314 (5804), 1442–1445 (2006).

Nikumbh, A. C., Chakraborty, A. & Bhat, G. S. Recent spatial aggregation tendency of rainfall extremes over India. Scientific reports 9 (1), 10321 (2019).

ADS   PubMed   PubMed Central   Google Scholar  

Varikoden, H., Revadekar, J. V., Kuttippurath, J. & Babu, C. A. Contrasting trends in southwest monsoon rainfall over the Western Ghats region of India. Climate Dynamics 52 (7–8), 4557–4566 (2019).

Bisht, D. S., Chatterjee, C., Raghuwanshi, N. S. & Sridhar, V. Spatio-temporal trends of rainfall across Indian river basins. Theoretical and applied climatology 132 (1–2), 419–436 (2018).

Sanikhani, H., Kisi, O., Mirabbasi, R. & Meshram, S. G. Trend analysis of rainfall pattern over the Central India during 1901–2010. Arabian Journal of Geosciences 11 (15), 437 (2018).

Paul, S. et al . Weakening of Indian summer monsoon rainfall due to changes in land use land cover. Scientific reports 6 , 32177 (2016).

ADS   CAS   PubMed   PubMed Central   Google Scholar  

Guhathakurta, P. & Rajeevan, M. Trends in the rainfall pattern over India. International Journal of Climatology: A Journal of the Royal Meteorological Society 28 , 1453–1469 (2008).

Srivastava, P., Singh, R., Tripathi, S. & Raghubanshi, A. S. An urgent need for sustainable thinking in agriculture–An Indian scenario. Ecological indicators 67 , 611–622 (2016).

Swain, S., Verma, M. & Verma, M. K. Statistical trend analysis of monthly rainfall for Raipur District, Chhattisgarh. Int. J. Adv. Engg. Res. Studies/IV/II/Jan.-March 87 , 89 (2015).

Chakraborty, S., Pandey, R. P., Chaube, U. C. & Mishra, S. K. Trend and variability analysis of rainfall series at Seonath River Basin, Chhattisgarh (India). International Journal of Applied Science and Engineering Research 2 (4), 425–434 (2013).

Mondal, A., Lakshmi, V. & Hashemi, H. Intercomparison of trend analysis of multisatellite monthly precipitation products and gauge measurements for river basins of India. Journal of Hydrology 565 , 779–790 (2018).

Bera, S. Trend analysis of rainfall in Ganga Basin, India during 1901-2000. American Journal of Climate Change 6 (1), 116 (2017).

Chandniha, S. K., Meshram, S. G., Adamowski, J. F. & Meshram, C. Trend analysis of precipitation in Jharkhand State, India. Theoretical and Applied Climatology 130 (1-2), 261–274 (2017).

Dash, S. K., Jenamani, R. K., Kalsi, S. R. & Panda, S. K. Some evidence of climate change in twentieth-century India. Climatic Change 85 , 299–321 (2007).

Patra, J. P., Mishra, A., Singh, R. & Raghuwanshi, N. S. Detecting rainfall trends in twentieth century (1871–2006) over Orissa State, India. Clim. Change 111 , 801–817 (2012).

Şen, Z. Innovative trend analysis methodology. Journal of Hydrologic Engineering 17 (9), 1042–1046 (2012).

Sen, Z. Trend identification simulation and application. J. Hydrol. Eng. 19 (3), 635–642 (2014).

Kisi, O. An innovative method for trend analysis of monthly pan evaporations. Journal of Hydrology 527 , 1123–1129 (2015).

Ay, M. & Kisi, O. Investigation of trend analysis of monthly total precipitation by an innovative method. Theoretical and Applied Climatology 120 (3–4), 617–629 (2015).

Martinez-Austria, P. F., Bandala, E. R. & Patiño-Gómez, C. Temperature and heat wave trends in northwest Mexico. Physics and Chemistry of the Earth, Parts A/B/C 91 , 20–26 (2016).

Onyutha, C. Identification of sub-trends from hydro-meteorological series. Stochastic Environmental Research and Risk Assessment 30 (1), 189–205 (2016).

Wu, H. & Qian, H. Innovative trend analysis of annual and seasonal rainfall and extreme values in Shaanxi, China, since the 1950s. International Journal of Climatology 37 (5), 2582–2592 (2017).

Şen, Z. Innovative trend significance test and applications. Theoretical and applied climatology 127 (3-4), 939–947 (2017).

Von Storch, H. Misuses of statistical analysis in climate research. Analysis of climate variability: applications of statistical techniques 2nd edn. In: Proceedings of an Autumn School organized by the Commission of the European Community on Elba from October 30 to November 6, 1993. Springer, Berlin pp 11–26 (1995).

Gedefaw, M. et al . Innovative trend analysis of annual and seasonal rainfall variability in Amhara regional state, Ethiopia. Atmosphere 9 (9), 326 (2018).

Kişi, Ö., Guimaraes Santos, C. A., Marques da Silva, R. & Zounemat-Kermani, M. Trend analysis of monthly streamflows using Şen’s innovative trend method. Geofizika 35 (1), 53–68 (2018).

Pandey, B. K. & Khare, D. Identification of trend in long term precipitation and reference evapotranspiration over Narmada river basin (India). Global and planetary change 161 , 172–182 (2018).

Darji, M. P., Dabhi, V. K. & Prajapati, H. B. Rainfall forecasting using neural network: A survey. IEEE International Conference on Advances in Computer Engineering and Applications Ghaziabad 2015 , 706–713 (2015).

Carter, J. G. et al . Climate change and the city: Building capacity for urban adaptation. Progress in planning 95 , 1–66 (2015).

Pal, S. & Talukdar, S. Modelling seasonal flow regime and environmental flow in Punarbhaba river of India and Bangladesh. Journal of Cleaner Production 252 , 119724 (2020).

Mukherjee, P., Singh, C. K. & Mukherjee, S. Delineation of groundwater potential zones in arid region of India—a remote sensing and GIS approach. Water resources management 26 (9), 2643–2672 (2012).

Chakraverty, S. & Gupta, P. Comparison of neural network configurations in the long-range forecast of southwest monsoon rainfall over India. Neural Comput Appl 17 , 187–192 (2008).

Şahin, M., Kaya, Y. & Uyar, M. Comparison of ANN and MLR models for estimating solar radiation in Turkey using NOAA/AVHRR data. Advances in Space Research 51 (5), 891–904 (2013).

Samantaray, S., Tripathy, O., Sahoo, A., & Ghose, D. K. Rainfall Forecasting Through ANN and SVM in Bolangir Watershed, India. In Smart Intelligent Computing and Applications 767-774 (2020).

Hossain, I., Rasel, H. M., Imteaz, M. A., & Mekanik, F. Long-term seasonal rainfall forecasting using linear and non-linear modelling approaches: a case study for Western Australia. Meteorology and Atmospheric Physics, 1–11 (2019).

Liu, Q., Zou, Y., Liu, X. & Linge, N. A survey on rainfall forecasting using artificial neural network. IJES 11 (2), 240–249 (2019).

Alam, W. et al . Improved ARIMAX modal based on ANN and SVM approaches for forecasting rice yield using weather variables. Indian Journal of Agricultural Sciences 88 (12), 101–105 (2018).

Arabeyyat, O., Shatnawi, N. & Matouq, M. Nonlinear Multivariate Rainfall Prediction in Jordan Using NARX-ANN Model with GIS Techniques. Jordan Journal of Civil Engineering 12 , 3 (2018).

Kala, A., & Vaidyanathan, S. G. Prediction of Rainfall Using Artificial Neural Network. In 2018 International Conference on Inventive Research in Computing Applications 339–342 (2018).

Mishra, S. K., & Sharma, N. Rainfall forecasting using backpropagation neural network. In Innovations in Computational Intelligence 277–288 (2018).

Şenkal, O., Şahin, M. & Peştemalci, V. The estimation of solar radiation for different time periods. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects 32 (13), 1176–1184 (2010).

Sahai, A. K., Soman, M. K. & Satyan, V. All India summer monsoon rainfall prediction using an artificial neural network. Climate dynamics 16 (4), 291–302 (2000).

Das, S., Dey, S., Dash, S. K., Giuliani, G. & Soloman, F. Dust aerosol feedback on the Indian summer monsoon: Sensitivity to absorption property. J. Geophys. Res. Atmos. 120 , 9642–9652 (2015).

Sharma, R., Hooyberghs, H., Lauwaet, D. & De Ridder, K. Urban heat island and future climate change—Implications for Delhi’s heat. Journal of Urban Health 96 (2), 235–251 (2019).

PubMed   Google Scholar  

Yue, S. & Hashino, M. Long term trends of annual and monthly precipitation in Japan 1. JAWRA Journal of the American Water Resources Association 39 (3), 587–596 (2003).

Yue, S. & Wang, C. Y. Applicability of prewhitening to eliminate the influence of serial correlation on the Mann‐Kendall test. Water resources research 38 (6), 4–1 (2002).

Von Storch, H. Spatial patterns: EOFs and CCA. In Analysis of climate variability (pp. 231–263). Springer, Berlin, Heidelberg (1999).

Gao, P., Mu, X. M., Wang, F., & Gao, P. Changes in streamflow and sediment discharge and the response to human activities in the middle reaches of the Yellow River (2011).

Zhang, S. & Lu, X. X. Hydrological responses to precipitation variation and diverse human activities in a mountainous tributary of the lower Xijiang, China. Catena 77 (2), 130–142 (2009).

Westra, S. et al . Future changes to the intensity and frequency of short‐duration extreme rainfall. Reviews of Geophysics 52 (3), 522–555 (2014).

Kumar, V. Optimal contour mapping of groundwater levels using universal kriging—a case study. Hydrological Sciences Journal 52 (5), 1038–1050 (2007).

Agrawal, R. & Srikant, R. Mining sequential patterns. In icde 95 , 3–14 (1995).

Vauclin, M., Vieira, S. R., Vachaud, G. & Nielsen, D. R. The Use of Cokriging with Limited Field Soil Observations 1. Soil Science Society of America Journal 47 (2), 175–184 (1983).

Nielsen, D. R., & Wendroth, O. Spatial and temporal statistics: sampling field soils and their vegetation. Catena Verlag (2003).

Wang, Y. Q. & Shao, M. A. Spatial variability of soil physical properties in a region of the Loess Plateau of PR China subject to wind and water erosion. Land Degrad. Dev. 24 (3), 296–304 (2013).

Duahn, D. & Pandey, A. Statistical analysis of long term spatial and temporal trends of precipitation during 1901–2002 at Madhya Pradesh, India. Atmospheric Research 122 , 136–149 (2013).

Taxak, A. K., Murumkar, A. R. & Arya, D. S. Long term spatial and temporal rainfall trends and homogeneity analysis in Wainganga basin, Central India. Weather and Climate Extremes 4 , 50–61 (2014).

Helsel, D. R. & Hirsch, R. M. Statistical methods in water resources 323, Reston, VA: US Geological Survey (2002).

Goyal, M. K. Statistical analysis of long term trends of rainfall during 1901–2002 at Assam, India. Water resources management 28 (6), 1501–1515 (2014).

Kisi, O. & Ay, M. Comparison of Mann–Kendall and innovative trend method for water quality parameters of the Kizilirmak River, Turkey. Journal of Hydrology 513 , 362–375 (2014).

ADS   CAS   Google Scholar  

Cui, L. et al . Innovative trend analysis of annual and seasonal air temperature and rainfall in the Yangtze River Basin, China during 1960–2015. Journal of Atmospheric and Solar-Terrestrial Physics 164 , 48–59 (2017).

Wickham, H. ggplot2: elegant graphics for data analysis. Springer (2016).

Du, J., Liu, Y., Yu, Y. & Yan, W. A prediction of precipitation data based on support vector machine and particle swarm optimization (PSO-SVM) algorithms. Algorithms 10 (2), 57 (2017).

Hong, W. C. Rainfall forecasting by technological machine learning models. Applied Mathematics and Computation 200 (1), 41–57 (2008).

Marumbwa, F. M., Cho, M. A. & Chirwa, P. W. Analysis of spatio-temporal rainfall trends across southern African biomes between 1981 and 2016. Physics and Chemistry of the Earth, Parts A/B/C 114 , 102808 (2019).

Sen Roy, S. A spatial analysis of extreme hourly precipitation patterns in India. Int. J. Climatol. 29 , 345–355 (2009).

Rajeevan, M., Bhate, J., Kale, J. D. & Lal, B. High resolution daily gridded rainfall data for the Indian region: Analysis of break and active monsoon spells. Curr. Sci. 91 (3), 296–306 (2006).

Mirza, M. Q., Warrick, R. A., Ericksen, N. J. & Kenny, G. J. Trends and persistence in precipitation in the Ganges. Brahmaputra and Meghna river basins. Hydrol. Sci. J. 43 , 845–858 (1998).

Wang, S. & Zhang, X. Long-term trend analysis for temperature in the Jinsha River Basin in China. Theor Appl Climatol 109 , 591–603 (2012).

Parthasarathy, B. & Dhar, O. N. Secular variations of regional rainfall over India. Q. J. R. Meteorol. Soc. 100 , 245–257 (1974).

Jain, S. K. & Kumar, V. Trend analysis of rainfall and temperature data for India. Current Science 102 (1), 37–49 (2012).

Guhathakurta, P., Rajeevan, M., Sikka, D. R. & Tyagi, A. Observed changes in southwest monsoon rainfall over India during 1901–2011. International Journal of Climatology 35 (8), 1881–1898 (2015).

Xu, C. et al . Decreasing Indian summer monsoon on the northern Indian sub-continent during the last 180 years: evidence from five tree-ring cellulose oxygen isotope chronologies. Climate of the Past 14 (5), 653–664 (2018).

Machiwal, D., Jha, M. K., Singh, V. P. & Mohan, C. Assessment and mapping of groundwater vulnerability to pollution: Current status and challenges. Earth-Science Reviews 185 , 901–927 (2018).

Guhathakurta, P. Long-range monsoon rainfall prediction of 2005 for the districts and sub-division kerala with artificial neural network. Curr Sci 90 , 773–779 (2006).

Khan, N. et al . Performance assessment of general circulation model in simulating daily precipitation and temperature using multiple gridded datasets. Water 10 (12), 1793 (2018).

Gosain, A. K., Rao, S. & Basuray, D. Climate change impact assessment on hydrology of Indian river basins. Current science 90 (3), 346–353 (2006).

Download references

Acknowledgements

The authors of the article would like to thank Indian Meteorological Department for providing the rainfall data of all sub-divisions to conduct this work. The authors also acknowledge the Department of Geography, University of Gour Banga, Malda, West Bengal and Department of Geography and Jamia Milia Islamia, New Delhi for providing laboratory facilities and other supports to carry out the research.

Author information

Authors and affiliations.

School of Humanities and Social Sciences, Indian Institute of Technology Indore, Simrol, Indore, 453552, India

Bushra Praveen & Pritee Sharma

Department of Geography, University of Gour Banga, Malda, India

Swapan Talukdar, Susanta Mahato & Jayanta Mondal

Department of Geography, Faculty of Natural Sciences, Jamia Millia Islamia, New Delhi - 110 025, India

Shahfahad & Atiqur Rahman

Department of Disaster management, Begum Rokeya University, Rangpur, 5400, Bangladesh

Abu Reza Md. Towfiqul Islam

You can also search for this author in PubMed   Google Scholar

Contributions

S.T. participated in the design of this study, and performed the statistical analysis, modeling and revise the manuscript. J.M. carried out data acquisition, data processing and some statistical analysis. A.R.M modeled cause of rainfall changes. Shahfahad wrote the initial draft, B.P. collected background information. S.T., Shahfahad, and S.M. carried out literature review. S.T., S.M., A.R.M., A.R., and Shahfahad reviewed and revised the manuscript. P.S. reviewed the manuscript. All authors have read and approved the content of the manuscript.

Corresponding author

Correspondence to Atiqur Rahman .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplemenatry information

Supplemenatry information., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Praveen, B., Talukdar, S., Shahfahad et al. Analyzing trend and forecasting of rainfall changes in India using non-parametrical and machine learning approaches. Sci Rep 10 , 10342 (2020). https://doi.org/10.1038/s41598-020-67228-7

Download citation

Received : 10 March 2020

Accepted : 06 May 2020

Published : 25 June 2020

DOI : https://doi.org/10.1038/s41598-020-67228-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Spatio-temporal epidemiology and associated indicators of covid-19 (wave-i and ii) in india.

• Karuppusamy Balasubramani
• Venkatesh Ravichandran
• Praveen Balabaskaran Nina

Scientific Reports (2024)

Long-term trends and spatial variability in rainfall in the southeast region of Bangladesh: implication for sustainable water resources management

• Hrithik Nath
• Sajal Kumar Adhikary
• Ahmed Ali A. Shohan

Theoretical and Applied Climatology (2024)

Linkages and reactions of geomorphic processes in Kerala Flood, 2018

• Satheesh Chothodi
• Sumesh Kuniyil
• Rahul Mishra

Natural Hazards (2024)

Times Series Forecasting of Monthly Rainfall using Seasonal Auto Regressive Integrated Moving Average with EXogenous Variables (SARIMAX) Model

• Shahenaz Mulla
• Chaitanya B. Pande
• Sudhir K. Singh

Water Resources Management (2024)

Modeling and forecasting rainfall patterns in India: a time series analysis with XGBoost algorithm

• Pradeep Mishra
• Abdullah Mohammad Ghazi Al Khatib
• Ramesh Yadav

Environmental Earth Sciences (2024)

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

Professional network data

Leverage our top B2B datasets

Job posting data

Get access to hundreds of millions of jobs

Employee review data

Get data for employee sentiment analysis

Enhanced professional network data

Employee data

Get data on global talent at scale

Funding data

Discover and analyze funding deals

Firmographic data

Unlock a 360° view of millions of companies

Technographic data

Analyze companies’ tech stacks

BY INDUSTRY

MOST POPULAR USE CASES

Company API

Find and get data on specific companies

Historical headcount API

See how company headcounts are changing

Employee API

Access millions of employee profiles

Jobs data API

Find relevant jobs with ease

Largest professional network

Company, employee, and jobs data

Company and jobs data

Company and review data

Company, jobs, review, salary data

Community and repository data

Leveraging web data for informed investing

Building or enhancing data-driven HR tech

Supercharging your lead generation engine

Transforming marketing with web data

Market research

Conducting comprehensive market research

Lead enrichment

Use Coresignal’s data for enrichment

Talent analytics

Analyze talent from multiple perspectives

Talent sourcing

Comprehensive talent data for recruitment

Investment analysis

Source deals, evaluate risk and much more

Target market analysis

Build a complete view of the market

Competitive analysis

Identify and analyze competitors

B2B Intent data

Lesser-known ways to find intent signals

Documentation

Detailed guides, samples, and dictionaries

Learn and get insipired

Find answers to popular questions

Resource center

Data insights, customer stories, expert articles

Trend Analysis: Types, Benefits, and Examples

Updated on December 5, 2023

Trend analysis is a strategy used in making future predictions based on historical data. It allows to compare data points over a given period of time and identify uptrends, downtrends, and stagnation.

If a trend is stable and steady over a period of time, it indicates consistency and invokes more certainty than a trend that is drastically changing positions. However, inconsistent trends might be more attractive to some investors who analyze certain external factors contributing to the radical trend changes. High risk usually involves chances of high rewards.

Investors and business managers use this information to make data-driven decisions and improve strategies.

Let’s find out more about trend analysis, its benefits, and real-life examples.

Types of trend analysis

Trend analysis is computed using numerical data. This information is usually historical data, either traditional data in the form of a company’s performance taken from its public financial statements or  public web data , such as the number of job postings of a competitor in the past five years. When adding numerical data to a chart, you will be able to identify three types of trends. 

Upward trend (bull market)

An uptrend or an upward trend means that your data points are increasing. Based on what type of variable you are examining and your purpose, this could have different meanings.

For instance, you are a business owner looking at the price of raw materials required to produce bread, and you notice that the price is increasing. This information could help you make different predictions, such as increased costs for your business or the necessity of raising the prices for the final consumer. 

At the same time, an investor looking at the share price of company X who noticed an upward trend might decide to buy the stock since the price is increasing. An upward movement in a stock’s price generally indicates a favorable condition, helping you to determine if the stock is a worthwhile investment. 

Downward trend (bear market)

On the opposite side, a downward trend indicates the decreasing value of your variable. For example, if a company’s profit has a sharp decline, this may require investors to proceed with caution as the stock is risky since the price is going down. This also applies when other economic or financial variables have a downward trend.

When investors  research financial assets , trend analysis can be done on the asset’s historical data. If this price is decreasing, it indicates the presence of a bearish market. In other words, investment is not recommended because the prices could further decrease, leading to a loss.

Horizontal trend

Finally, the horizontal line indicates stagnation. In other words, the prices, or any other metrics, are not going up or down; rather, they are stagnant.

In practice, a flat trend might go up for one period, then pull a trend reversal, reaching a steady general direction overall. Making investment decisions based on horizontal trends is risky because you do not know what will happen. However, if you decide to go with it, a sophisticated revenue and cost analysis regarding the sales regions must be implemented to calculate the risks.

Perform trend analysis with fresh web data

Leverage historical data to identify potential investment opportunities.

Limitations

Identifying turning moments is a major issue in trend forecasting. Turning points are obvious in retrospect, but it can be difficult to identify whether they are simply deviations or the start of new trends at the time.

Long-term estimates require additional data, which may not always be available. Especially for a new business or product line. In any event, the further out one anticipates, the higher the risk of mistakes, because time inevitably introduces new variables.

As a result, it's crucial to examine your trend analysis data and take action only if you're confident in your market reading.

Key caveats of any trend analysis include recognition that prior trends do not always continue into the future. Also, short-term linear trends may actually be non-linear over longer periods, plus long-term linear trends may have short-term cycles.  Finally, trend analyses are lousy at picking up black-swans or even slightly-grey-swans.

- John A. Kilpatrick, Ph.D. MAI, Greenfield Advisors

Benefits of trend analysis

Apart from being a straightforward investment analysis tool, trend analysis has several other benefits. Some of the main ones include:

• It is easy to compare the performance of two or more firms over the same period of time, so you can see how strong or weak a business is compared to another one in the same industry.
• Trend analysis can be used with a myriad of numerical data types , including traditional data (i.e., profit or expenses) and public web data (website traffic, customer complaints, POS transactions, and many more). 
• Data suggests you can use these long-term trends to identify actionable patterns . These patterns can afterward be used to make forecasts.
• You can use trend analysis to examine preliminary financial statements for inconsistencies and see whether certain adjustments must be implemented before releasing the statements to the public.
• Trend analysis allows you to examine the entire stock market to detect signs of potential trend changes for better or worse.

The core benefit of the trend analysis is that you can compare your incoming data with another firm's and measure your firm's performance in a realistic way. If you know the exact way to analyze the trend then you’ll be able to identify which direction your business is going.

- Larry Hart,  The Stock Dork

Don’t miss a thing

Subscribe to our monthly newsletter and receive the summary of our latest publications, data guides, and other news straight to your inbox.

By providing your email address you agree to receive newsletters from Coresignal. For more information about your data processing, please take a look at our Privacy Policy .

What are the tools used for trend analysis?

Preparation.

In order to do trend analysis, you must decide on what segment, industry, or even asset you want to use. For example, you may want to look at the bond market.

Once you make this decision, you also need to determine the period. There is no consensus on the actual amount of time for the movement to be considered a trend.

As a result, this depends on the historical data available and your purposes. 

Trend analysis tools

There are numerous management tools for trend analysis. One of the most basic ones is to simply plot the data points and visually establish the presence of a trend. For example, one of the most popular data plotting tools is Tableau which allows you to visualize the data through graphs, charts, and other models. Another option is to transform this data into  moving averages  that will eliminate fluctuations for better trend identification.

As a result, you must have access to the following:

• Raw numerical data 
• Access to analysis software 

One of the most useful management tools for trend analysis is Google trends. It allows you to discover what people search for by entering a keyword into the search engine.

Trend analysis examples

Trend analysis that uses business information can be useful for both managers and stakeholders, including potential investors. For instance, you can perform a trend analysis using public web data, such as website traffic for any given company. 

The figure below shows the total website traffic in the last six months for company A, an online store that sells gifts. Data suggests an uptrend during the holiday season, reaching the peak on the 20th of December. 

After the first half of January, there has been a relatively horizontal trend. In other words, if you had a competing gift store, you could compare your performance to this company. Although intuitive, this example of trend analysis helps you predict future results and performance or compare this company to a competitor’s activity.

One of the most common trend analysis strategies is when you are examining the share price of a financial asset to help with the decision-making process. For example, the figure below compares the share prices of two companies, X and Y, over one year.

Company X shows an overall uptrend over the past year with a small trend reversal in February. However, company Y had a horizontal trend for the first half months, after which it started to decrease.

Generally, investors are more cautious when there is a horizontal trend because it is difficult to forecast when the price will change its direction and whether it will be up or down. In this case, the share price has a steady decrease, which will result in a loss if added to your portfolio. 

Trend analysis is only as good as the information you have available. And even if you believe to have the most accurate information available, statistical noise along with randomness will always be present to distort your results. Therefore, you have to be very objective about your results and not let your sentiments drive your decisions. Furthermore, you have to combine different analytical techniques since no one method will provide you the most accurate result.

Alex Williams, CTO of  FindThisBest

Company X’s increasing trend might help you predict future events and indicate that this stock is a great addition to an investor’s portfolio, especially if you have a long-term investment strategy.

However, other information should also be considered when performing a trend analysis, both related to the company itself and the overall market and the economy. Trend analysis is only one tool that investors can use to identify the profitability of a given asset. 

Trend analysis using Coresignal's historical data

You can also perform trend analysis by leveraging our public web data. For example, our historical headcount data allows you to see employee number changes over time in a specific company. In investing, it provides you with valuable insights into the company's growth and longevity.

In the figure above, you can see Tesla's headcount change from September 2018 to November 2021. From this graph, you can try to make sense of what happened during the transition from the end of 2019 to the beginning of 2020 that caused a drop in headcount.

Going further, you see that the employee number kept growing steadily and consistently. You can make a prediction that the trend will keep increasing at a steady rate unless the same thing that happened at the end of 2019 happens again.

As an investor, you would need to perform an analysis and figure out what caused the drop and whether the company has implemented prevention methods to keep that from happening again.

We offer historical data starting from 2018. The longer the time period, the more notable the trend. With up to 5 years of historical data, you can analyze if and how seasons, certain political events, and other ESG factors affect a company's performance.

In general, trend analysis is extremely valuable for investors and business owners. Considering the current data availability, the value of trend analysis is inseparable from data-driven decisions, especially while leveraging  public web data .

Public web data allows you to perform a more in-depth analysis, in turn outsmarting a part of your competition that is not using external data to their advantage. Data-driven trend analysis is also a great way of anticipating future events that could enhance your investment intelligence and find better business opportunities.

Stay ahead of the game with fresh web data

Coresignal's data helps companies achieve their goals

Boost your growth

See a variety of datasets that will help your business growth.

Related articles

Sales & Marketing

10 Most Reliable B2C and B2B Lead Generation Databases

Not all lead databases are created equal. Some are better than others, and knowing how to pick the right one is key. A superior...

Mindaugas Jancis

April 23, 2024

It’s a (Data) Match! Data Matching as a Business Value

With the amount of business data growing, more and more options to categorize it appear, resulting in many datasets....

April 9, 2024

Data Analysis

Growing demand for sustainability professionals 2020–2023

Original research about the changes in demand for sustainability specialists throughout 2020–2023....

March 29, 2024

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Trend analysis for national surveys: Application to all variables from the Canadian Health Measures Survey cycle 1 to 4

Yi-sheng chao.

1 Centre de recherche du centre hospitalier de l’Université de Montréal (CRCHUM), Université de Montréal, Montréal, Québec, Canada

Chao-Jung Wu

2 Département d'informatique, Université du Québec à Montréal, Montréal, Québec, Canada

Hsing-Chien Wu

3 Taipei Hospital, Ministry of Health and Welfare, New Taipei city, Taiwan

Wei-Chih Chen

4 Department of Chest Medicine, Taipei Veterans General Hospital, Taipei, Taiwan

5 Faculty of Medicine, School of Medicine, National Yang-Ming University, Taipei, Taiwan

Associated Data

The data underlying this study are CHMS data belonging to Statistics Canada. The authors did not have any special access to the CHMS data. It is against the Statistics Act of Canada to release data that are de-identified. The CHMS data is available through the Research Data Centres program administered by Statistics Canada (see this link for eligibility and detailed process to request access: https://www.statcan.gc.ca/eng/rdc/index ). Data access needs to be approved by Statistics Canada, and any analysis output needs to be vetted by Statistics Canada before being released.

Trend analysis summarizes patterns over time in the data to show the direction of change and can be used to investigate uncertainties in different time points and associations with other factors. However, this approach is not widely applied to national surveys and only selected outcomes are investigated. This study demonstrates a research framework to conduct trend analysis for all variables in a national survey, the Canadian Health Measures Survey (CHMS).

Data and methods

The CHMS cycle 1 to 4 was implemented between 2007 and 2015. The characteristics of all variables were screened and associated to the weight variables. Missing values were identified and cleaned according to the User Guide. The characteristics of all variables were extracted and used to guide data cleaning. Trend analysis examined the statistical significance of candidate predictors: the cycles, age, sex, education, household income and body mass index (BMI). R (v3.2) and RStudio (v0.98.113) were used to develop the framework.

There were 26557 variables in 79 data files from four cycles. There were 1055 variables significantly associated with the CHMS cycles and 2154 associated with the BMI after controlling for other predictors. The trend of blood pressure was similar to those published.

Trend analysis for all variables in the CHMS is feasible and is a systematic approach to understand the data. Because of trend analysis, we have detected data errors and identified several environmental biomarkers with extreme rates of change across cycles. The impact of these biomarkers has not been well studied by Statistics Canada or others. This framework can be extended to other surveys, especially the Canadian Community Health Survey.

Trend analysis that summarizes the patterns across time has been popularly used in a variety of disciplines, such as business[ 1 ], financial market[ 2 ], economics[ 3 ] and epidemics or mortality[ 4 – 7 ]. Trend analysis helps to estimate the quantities of current or previous events and their variability or uncertainties in different time points. It is also the foundation for prediction and projection after analyzing the significance of time and relationships with other predictors[ 8 – 10 ]. For national surveys, certain trends have been studied to show the progress or deterioration in public health and health care[ 11 ]. These trends provide important clues for the healthcare professionals to understand the unmet needs for care and the magnitudes of health problems. The comparison of multiple trends allows us to prioritize the issues and allocate resources[ 4 , 12 ]. If well conducted, projections can be made to further prepare incoming challenges to health systems[ 8 , 9 ].

However, there are certain issues arising if taking this approach. First, the adjustment of survey design requires researchers to assign appropriate weights and specify survey sampling units and strata[ 13 ]. The identification of the necessary variables requires extra attention and expert knowledge. Second, the adjustment of survey design also limits the options of research tools[ 14 ]. The automatic procedures developed for time series data or repeated surveys are not applicable concerning survey design[ 1 ]. Linear methods, such as generalized linear models and principal component analysis, remain useful for surveys to generate nationally representative statistics[ 14 , 15 ].

Third, the access to the data may be restricted. For example, some of the Statistics Canada data products can be accessed only through the Research Data Centres (RDC) for academic researchers, such as the Canadian Health Measures Survey (CHMS)[ 16 ]. Physical restrictions may prevent complicated or exhaustive research protocols from being conducted for researchers outside Statistics Canada or other collaborating agencies. Fourth, the outcomes analyzed in national surveys are often limited to individuals’ interests. There are many published studies conducted trend analysis of the CHMS data but only limited numbers of variables are taken as target for analysis, especially hypertension and obesity related factors[ 17 – 23 ]. Even if trends are studied by data holders or affiliated researchers, important issues may remain unanswered. For example, the extensive review of environmental chemicals by Health Canada is not informative because statistics are listed by cycle without testing the significance of time trends or association with other contextual factors[ 24 – 26 ]. This needs to be addressed because effective use or extensive application of trend analysis to national surveys may lead to more efficient biomonitoring[ 11 ] and better identification of unexpected disease trends[ 17 ].

Four, trend analysis may impose challenges to computing resources[ 27 , 28 ]. The large numbers of variables in national surveys may limit the use of this method if not well planned. Lastly, there may not be sufficient incentive for academia, especially the researchers mainly funded by research grants, to innovate toward novel objectives in the long run[ 29 ]. Trend analysis with national surveys requires exhaustive research on documentation and survey method beforehand. There is no immediate benefit by studying variables other than the outcomes that are related to or can lead to research funding.

To address these issues that may be encountered while conducting trend analysis with national surveys, this study aims to 1) propose a framework of trend analysis for all variables in national surveys developed based on the CHMS data, 2) test the feasibility of trend analysis with all CHMS variables using computing resources available to most researchers, 3) summarize the results of the research framework and compatibility with previous studies, and 4) describe some of the obstacles and issues that may be encountered if applied to other surveys.

There were several major steps designed to execute this framework with the CHMS data after reviewing the data structure, data dictionaries, the CHMS User Guide[ 30 , 31 ] and the CHMS Cycle 1 to 8 Content Summary[ 32 ]. This framework was applied to the CHMS data to generate a customized research flowchart in Fig 1 . First, all variables were imported from data files and screened for basic characteristics, including file names, variables of weights, bootstrap weight files to be merged, maximal and minimal values, responses and variable types (continuous or categorical). For the CHMS variables, the maximal values were important for data cleaning because the missing values were always coded with values far exceeding the observed values[ 30 , 31 , 33 ]. The values ending in 4, 5, 6, 7, 8, and 9 might represent “values higher than limits of detection”, “values less than limits of detection”, “not applicable”, “don’t know”, “refusal” and “not stated”[ 30 , 31 , 33 ]. For other surveys, missing values might be represented with certain values[ 34 ] or be coded with reserve values, such as -1 to -3[ 35 ]. To prevent computer memory from being exhausted, the data sets were always removed from the memory if unused.

Second, user-defined summary variables were be generated once data was stored for cleaning. The summary variables remained blank at this stage and could be the summaries of medication use, biomarker abnormality, or numbers of chronic conditions, depending on the research objectives. After these two steps, an exhaustive list of the CHMS variables was created. Original and derived variables were listed together and could be important indicators of data processing quality. An illustration of the variable list was shown in Table 1 .

Third, the CHMS data were cleaned based on the reserve values ending in four to nine[ 30 , 31 , 33 ]. The problem particular to biomarker data was that there were values larger or less than the upper or lower limits of detection. Health Canada imputed the values less than the limits of detection with half of the limits of detection[ 24 – 26 ]. In addition, Health Canada excluded the variables with more than 40% of subjects having values less than limits of detection from analysis[ 24 – 26 ]. In contrast, there were currently no official guide to impute values larger than the upper limits of detection and were tentatively imputed with 110% of the upper limits of detection.

Fourth, the summary variables or the derived ones needed to be recoded or calculated after data cleaning. For example, the summary variables of medication use included the use and the numbers of prescription drugs for cardiovascular conditions. This needed to be derived from the drug codes, either Anatomical Therapeutic Chemical (ATC) Classification System or American Society of Health-System Pharmacists (ASHP) drug codes[ 36 ]. Another example was that the chronic conditions reported in the CHMS could be further simplified or summarized in the numbers of chronic conditions diagnosed. Abnormality of disease biomarkers could be identified through external information, such as the clinical reference ranges used by health professionals[ 37 , 38 ]. The numbers of abnormality in biomarkers could be derived after data labeling. Certain secondary biomarkers, such as the estimated creatinine clearance that is used to evaluate kidney health[ 39 , 40 ], could also be derived after data cleaning.

In addition, some of the original variables needed to be made consistent across the CHMS cycles. The inconsistency arose for a variety of reasons, such as the changes in the measurement sample (serum or plasma), whether subjects fasted or not, and categorization of continuous variables. For example, the level of glucose was measured with plasma in the CHMS cycle 1 and with serum in the other cycles. In cycle 3 and 4, glucose was only quantified with fasted subjects. The glucose measurement with serum or plasma could be taken compatible[ 41 ] and could be recoded to the same variable. However, the fasted glucose levels had different diagnostic values from those not fasted and needed to be distinguished[ 42 – 44 ]. Therefore, glucose measured with serum or plasma among fasted and non-fasted subjects were recoded to two variables that represented fasted glucose in cycle 3 and 4 and non-fasted in cycle 1 and 2.

Fifth, some of the summary or derived variables needed to be merged to other data sets to obtain useful statistics. For example, the file of medication use in the CHMS cycle 3 was not assigned survey weights and needed to be merged with the household or other data files to understand issues such as prevalence of drug use or numbers of prescription drugs. The other example was that the information on non-environmental biomarkers in cycle 3 was stored in a stand-alone data set with identification numbers that could be used for data merging. In such cases, the summary variables of medication or abnormality in clinical biomarkers were generated in respective data files and merged to household data files for inference.

Sixth, descriptive or analytical study of all CHMS variables could be conducted. In this study, trend analysis was performed with the CHMS cycles in continuous scales as the only predictor to understand whether there were significantly increasing or decreasing trends across cycles. It was also possible to add more predictors that were important for researchers, such age, sex and provinces. Continuous and binary outcomes were analyzed with linear and logistic regression respectively. The sample sizes, model fit statistics, p values of predictors and variance inflation factors of all predictors were obtained. However, there were several issues to be dealt with for the adjustment of survey design. The sample sizes should be sufficient relative to the primary sampling units. For the CHMS, the sampling units were the cities of clinical visits[ 30 , 31 , 33 ]. The numbers of unweighted sample sizes should satisfy the vetting rules administered by Statistics Canada, which varied by survey and analytical method. The collinearity issue could be assessed between predictors[ 45 ]. To avoid memory overload and increase computation efficiency, only necessary variables were loaded for regression analysis. Lastly, the results were reorganized for vetting and release. The trends were plotted against the CHMS cycles along with the necessary summary tables designated for release vetting by the RDC analyst.

Age[ 46 ] and blood pressure[ 47 ] that had official statistics released were the examples of trend analysis using the CHMS data. The trends were illustrated in relative values compared to the mean values in the CHMS cycle 1. The 95% CIs (confidence intervals) were plotted as shade areas. The details in the blood pressure measurement could be found elsewhere[ 48 , 49 ]. The significance of time trends was confirmed if there was significant association with the CHMS cycles in continuous scale based on linear regression adjusting for survey design. The association with body mass index (BMI) was also tested with linear regression, while age in years, sex, household income in Canadian dollars, and educations in four categories (less than secondary school education, secondary school education, some post-secondary, and post-secondary graduation) were controlled. BMI was calculated as weight in kilograms divided by height in meters squared[ 15 , 50 ]. This study was conducted at the Research Data Centre (RDC) at McGill University (Montréal, Québec, Canada). The computer at the RDC was equipped with Intel i7 3070 CPU (central processing unit, 4 cores 8 thread), 16 GB RAM (Random-access memory), 128 GB SSD (solid state disk) and an operating system, Window 7 Professional 64 bit (Microsoft Corporation, Seattle, USA). Data processing and analysis were conducted with R (v3.20)[ 51 ] and RStudio (v0.98.113)[ 52 ]. Biomarkers were the variables that were identified in the CHMS Cycle 1 to 8 Content Summary[ 32 ]. This Summary also defined environmental biomarkers that were the chemicals that could be detected in human specimens or living spaces Statistics Canada, 2015 #451}. P values, two-tailed, less than 0.05 were considered statistically significant. The processing time was reported to help researchers understand the complexity of trend analysis using national surveys.

Data processing and analysis

There were 26557 original variables in 79 data files released before March 2017. In 32 data files, 16064 variables were related to bootstrap weights only. There were 19212 variables created to summarize data or derived to represent important secondary outcomes for future projects. Using a typical desktop computer at McGill RDC, the processing time of each major step was estimated in Fig 1 . First, the data were imported from STATA format and then stored in R data format. Data importation, storage and screening took less than five minutes to finish. In the third step, the cleaning of all original variables took less than 30 minutes. However, the creation of the summary measures or derived secondary outcomes in the fourth step, such as the numbers of chronic conditions, medication use, and abnormality in biomarkers, was time-consuming. The processing time could be up to two days. At least two factors were contributing to the long processing time. The first factor was that efficient variable-wise calculation was not applicable. Depending on the nature of derived variables, there might be subject-based operation and each observation needed to be screen, for example, for the numbers of cardiovascular or diabetes medication for each individual. The other factor was due to time spent on loading data to memory and writing processed data back to disk.

In the fifth step, the summary or derived variables that needed to be linked to or reproduced in other data files, such as the information on medication use and biomarker summaries, were merged to destination files. For example, the summary of medication use needed to be merged to the household data set and used with appropriate bootstrap weights to obtain nationally representative statistics. This took less than one hour to finish. Sixth, trend or regression analysis with and without the adjustment of other predictors took less than one day to finish for all original or derived variables. The predicted values of all CHMS variables could also be calculated within one day. Lastly, selected trends and summary tables were produced for vetting and release from the RDC within 10 minutes. This research framework took less than four days to screen and analyze all CHMS variables.

Characteristics of the CHMS cycles and Canadians

The summary of the CHMS data and the population characteristics were shown in Table 2 . The cycle 3 had the most numbers of variables and many of them were ever repeated in other cycles. There were cycle-4 variables to be released after April 2017. In cycle 2, there were more biomarkers than in any others. Because of the large numbers of biomarkers in cycle 2 and 3, there were variables designed to label limits of detection for all subjects.

The numbers of Canadians increased over time, from 29 to 32 million between cycle 1 and 4. About half of them were female. The proportion of females may not be different from that obtained with other data sources[ 53 ]. The minimal ages were three years in cycle 1 and six in cycle 2 to 4. The maximal ages were 79 for all cycles. The mean age remained similar and might not be different from the official statistics, which described age by median values[ 46 ]. The ranges of blood pressure might also be similar to those published based on the same data[ 47 ], while Canadians of all ages were included in this study. In Fig 2 , the trends of age, mean arterial pressure, and systolic and diastolic blood pressure were shown along with their 95% confidence intervals (CIs) compared to the first measures in the CHMS. None of the trends was significantly associated with the CHMS cycles (p> 0.05 for all). Age and blood pressure were significantly associated with BMI while controlling for age, sex, education and household income (p <0.05 for all).

*significantly associated with body mass index (p<0.05).

Summary of the trends in the CHMS data

In Table 3 , the findings of the trend analysis were summarized. In the first row, the numbers of the CHMS variables that had been repeatedly measured were listed. There were 519 variables measured in CHMS cycle 1 to 4. The rates of change of BMI from cycle 1 to 4 were listed. There were 429 variables significantly associated with the CHMS cycles from one to four and 86 of them were biomarkers identified by Statistics Canada (p<0.05 for all). There were 1099 variables significantly associated with BMI and 152 of them were biomarkers. There were 20 and 26 variables respectively increasing and decreasing for more than 10% in three time intervals from cycle 1 to 4. There were 52 and 68 biomarkers observed to respectively increase or decrease once for at least 10% from cycle 1 to 4. Compared to the average growth rates of BMI, 0.2% per cycle, there were 130 biomarkers increasing more rapidly and 22 of them were non-environmental biomarkers.

There are large numbers of the CHMS variables and biomarkers increasing or decreasing at high rates. The importance of these trends to public health and wellbeing are not clear because current rate of investigating and publishing the trends of the CHMS variables is not satisfying. There were less than ten trends of the CHMS variables published between 2015 and 2017 including those only considering selected populations[ 48 , 54 , 55 ]. It can take more than ten years to have a comprehensive understanding in the trends of the biomarkers or physical activities or other variables, given the large numbers of variables in national surveys. Currently the CHMS data have been mostly used as a novel data source[ 12 , 18 – 23 , 56 , 57 ], rather than a continuous effort to monitor population health. Only several outcomes have been studied continuously among selected populations[ 48 , 54 , 55 ], in addition to the biomonitoring activities by Health Canada[ 24 – 26 ].

This research framework of trend analysis customized to the CHMS data is highly feasible with computing resources available to most researchers. Scaling up trend analysis to all variables in national surveys has several advantages. In the first place, the automated data cleaning system is effective and efficient. It takes less than 30 minutes to clean all 79 files from the CHMS cycle 1 to 4. The results of data cleaning are examined based on parameters such as the maximal or minimal values to ensure appropriate quality for subsequent trend analysis. Another advantage is that the visualization of trends is easy to understand and useful to prioritize biomarkers or variables for evaluation. In this study, the trends of blood pressure is plotted with the BMI trend to contrast the different patterns. We are applying this method to other variables to find unexpected trends. Moreover, certain types of data errors can also be easily highlighted with the trends. For example, the measurement unit of blood fibrinogen is mislabeled and leads to more than 10-fold decrease in the levels after the CHMS cycle 2 (personal communication with Statistics Canada). The trends with the highest and lowest rates of increase or decrease are easy targets for data quality examination.

Finally, this framework of trend analysis can be supplemented with regression analysis, prediction and projection subsequently. Multiple regression for all CHMS variables to identify the significance of BMI and socioeconomic status has been tried and proven realistic. Predicted values are retrieved to understand the trends least explained by BMI and socioeconomic status (statistics not requested for release). The CHMS has also been used for the projection of obesity trends[ 10 ] and projection is also possible.

Limitations

However, there are several limitations to the research framework. First, there may be other data or documentation errors not identified. The data and documentation accuracy of several of the trends of the largest relative magnitude of change have been confirmed (personal communication with Statistics Canada). There may be other errors that cannot be identified with trend analysis. The other issue is that the imputation method for right- or left-censoring can be improved. Health Canada imputes censored environmental chemicals according to the limits of detection and proportions of subjects within the limits[ 24 , 26 , 58 ]. Other advanced methods may be tried to take other contextual factors into consideration[ 59 , 60 ]. In fact, it is unclear whether the proportions used by Health Canada are based on weighted or unweighted statistics[ 24 , 26 , 58 ]. This study uses unweighted proportions to exclude the variables from analysis.

Furthermore, the codes have been written inside the RDC and suffered from significant time and resource constraints. The research framework will be structured into an R package for application to other major surveys and research purposes. There are several improvements expected for the implementation. For example, the evaluation of data products can be customized and made interactive. The method to create a list of variable characteristics to be extracted is related to the research hypothesis and should be made flexible for other projects. The introduction of external information to create or derive new variables as predictor or outcome can be improved. We are introducing the reference ranges for clinical or disease biomarkers[ 37 , 38 ] to further interpret clinical data and population health status. A system that describes the relationships between variables to infer information between them will be useful for sequential questions that study complicated status, such as disease history or evolution of life events. We are also considering incorporating imputation of missing information into the research framework[ 60 ].

Extension to other surveys

This research framework can be extended to other major surveys with similar data structure, variable naming systems, missing value identification strategies and sampling frames, especially the Canadian Community Health Survey[ 48 , 56 ]. For other major surveys that provide cleaned data[ 61 ] or do not use bootstrap weights[ 35 ], it requires minimal revision to replicate this research framework to conduct trend analysis for all variables. The automated process for visualization of trend analysis is suggested for researchers to look for neglected trends and for survey administrators to search and correct data errors that can be demonstrated with trends of extreme rates of change across cycles or time points.

Declaration

Ethics review.

This secondary data analysis was approved by the ethics review committee at the Centre Hospitalier de l’Université de Montréal.

Acknowledgments

The analysis presented in this paper was conducted at the Quebec Interuniversity Centre for Social Statistics, which is part of the Canadian Research Data Centre Network (CRDCN). The services and activities provided by the QICSS are made possible by the financial or in-kind support of the Social Sciences and Humanities Research Council (SSHRC), the Canadian Institutes of Health Research (CIHR), the Canada Foundation for Innovation (CFI), Statistics Canada, the Fonds de recherche du Québec—Société et culture (FRQSC), the Fonds de recherche du Québec—Santé (FRQS) and the Quebec universities. The views expressed in this paper are those of the authors, and not necessarily those of the CRDCN or its partners[ 16 ].

Funding Statement

Funded by Fonds de Recherche du Québec - Santé (CA) Postdoctoral fellowship to Yi-Sheng Chao. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability

Publication trends and research hotspots of the cardiorenal syndrome: A bibliometrics and visual analysis from 2003 to 2023

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Open the PDF for in another window
• Get Permissions
• Cite Icon Cite
• Search Site

Yibo Shi , Zean Fu , Shixiong Wu , Xinyi Yu; Publication trends and research hotspots of the cardiorenal syndrome: A bibliometrics and visual analysis from 2003 to 2023. Cardiorenal Med 2024; https://doi.org/10.1159/000539306

Download citation file:

• Ris (Zotero)
• Reference Manager

Article PDF first page preview

Introduction: Cardiorenal syndrome encompasses a range of disorders involving both the heart and kidneys, wherein dysfunction in one organ may induce dysfunction in the other, either acutely or chronically. This study conducted a literature search on cardiorenal syndrome from January 1, 2003, to September 8, 2023. Meanwhile, a quantitative analysis of the developmental trajectory, research hotspots and evolutionary trends in the field of cardiorenal syndrome through bibliometric analysis and knowledge mapping. Summary: The field of cardiorenal syndrome exhibits promising potential to grow and is emerging as a prominent research area. Future endeavours should prioritise a comprehensive understanding of the field and foster multi-centre co-operation among different countries and regions. Key Messages: The annual publication trend analysis revealed a consistent annual increase in cardiorenal syndrome literature over the last 20 years. The IL6, REN and INS genes were identified as the current research hotspots.

Email alerts

Citing articles via, suggested reading.

• Online ISSN 1664-5502
• Print ISSN 1664-3828

INFORMATION

• Contact & Support
• Information & Downloads
• Rights & Permissions
• Terms & Conditions
• Catalogue & Pricing
• Policies & Information
• People & Organization
• Stay Up-to-Date
• Regional Offices
• Community Voice

SERVICES FOR

• Researchers
• Healthcare Professionals
• Patients & Supporters
• Health Sciences Industry
• Medical Societies
• Agents & Booksellers

Karger International

• S. Karger AG
• P.O Box, CH-4009 Basel (Switzerland)
• Allschwilerstrasse 10, CH-4055 Basel
• Tel: +41 61 306 11 11
• Fax: +41 61 306 12 34
• Contact: Front Office
• Experience Blog
• Privacy Policy
• Terms of Use

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Moving Trend Analysis Methodology for Hydro-meteorology Time Series Dynamic Assessment

• Open access
• Published: 15 May 2024

Cite this article

You have full access to this open access article

• Zekâi Şen 1  

In the last 30 years, there are many publications in the literature due to global warming and climate change impacts exhibiting non-stationary behaviors in hydro-meteorology time series records especially in the forms of increasing or decreasing trends. The conventional trend analyzes cover the entire recording time with a single straight-line trend and slope. These methods do not provide information about up and down partial moving trends evolution at shorter durations along the entire record length. This paper proposes a dynamic methodology for identifying such evolutionary finite duration moving trend method (MTM) identifications and interpretations. The purpose of choosing MTM was to investigate the dynamic partial trend evolution over the recording period so that dry (decreasing trend) and wet (increasing trend) segments could be objectively identified and these trends could assist in water resources management in the study area. The moving trend analysis is like the classical moving average methodology with one important digression that instead of arithmetic averages and their horizontal line representations, a series of finite duration successive increasing and decreasing trends are identified over a given hydro-meteorology time series record. In general, partial moving trends of 10-year, 20-year, 30-year and 40-year occur above or below the overall trend and thus provide practical insight into the dynamic trend pattern with important implications. The moving trend methodology is applied to annual records of Danube River discharges, New Jersey state wise temperatures and precipitation time series from the City of Istanbul.

Avoid common mistakes on your manuscript.

1 Introduction

Global warming due to greenhouse gas (GHG) released into the troposphere causes climate change, which can reflect itself as partial and overall increasing or decreasing trends in any hydro-meteorological time series. It has already been proven by Milly et al. ( 2008 ) that all hydro-meteorological records fall into non-stationary realm with the inclusion of trends. Hydro-meteorological records have systematic components such as trends and variability (Burn and Hag Elnur 2002 ). There are basic methodological contributions that provide probabilistic or statistical trending tests (Mann 1945 ; Kendal 1970 ; Spearman 1904 ; Sen 1968 ; IPCC 2007 ; Şen 2012 ; Alashan 2018 ; Güçlü 2018 ; Ashraf et al. 2021 ). The applications of these methods have increased in an unprecedented way and therefore there are many studies in different disciplines that try to determine the overall (holistic) trend. Hamed ( 2008 ) has proposed different trend detection methodologies in hydro-meteorology records. A review of current trend methodologies is presented by Sonali and Kumar ( 2013 ) and applied these methodologies to temperature records. One of the major problems in the Mann and Kendall (MK) trend identification methodology is the assumption that time series must be serially independent, which cannot be met with hydro-meteorological data and hence pre-whitening (Yue and Wang 2002 ) or over-whitening (Şen 2017 procedures are offered to alleviate this requirement. In different water related disciplines many researchers applied the MK methodology holistically (overall) on a given time series records (Hirsch et al. 1982 ; van Belle and Hughes 1984 ; Hirsch and Slack 1984 ; Cailas et al. 1986 ; Hipel et al. 1988 ; Demaree and Nicolis 1990 ; Yu et al. 1993 ; Gan 1998 ; Taylor and Loftis 1989 ; Lins and Slack 1999 ; Douglas et al. 2000 ; Hamilton et al. 2001 ; Kalra et al. 2008 ; O’Brien et al. 2021 ; Mateus and Pitoto 2022 ).

The averaging of a partially fixed number of sub-time series is the basis of the moving average procedure, which can be weighted or unweighted. The average amount moves over time, repeatedly deleting old data points, leaving the average levels in that order. The unweighted alternative is a simple moving average procedure, where all observations are given the same weight. The weighted moving average, on the other hand, assigns different weights in the mean calculations. The basic idea in weights is that the newest data has more weights for prediction than the old ones, so exponentially increasing weights can be added towards the newest records. Such assignments may dependent on expert opinions, for example Yeh et al. ( 2003 ) proposed exponential weighted moving average control charts to detect small changes in process variability. Lee and Apley ( 2011 ) proposed another graph for the design of the residual-based weighted moving average for autoregressive moving average (ARMA) models (Box and Jenkins 1976 ).

The unweighted moving average method has advantages such as easy understanding and computation, small data requirement from the past, and removal of outliers after certain periods. On the other hand, disadvantages include that the estimate depends on the average length, requires all data from the past and gives the same weight or importance to all data used in the calculations.

The moving trend method (MTM) in this paper is based on the application of the innovative trend analysis (ITA) procedure proposed by Şen ( 2012 ) and used by several authors for hydro-meteorological datasets. Recently, ITA methodology is used along with traditional trend identification methodologies by many authors (Achite et al. 2021 ; Esit et al. 2021 ; Ullah et al. 2022 ; Hirca et al. 2022 ; Fanta et al. 2022 ; Pastagia and Mehta 2022 ; Xie et al. 2022 ; Abbas et al. 2023 ; Birpınar et al. 2023 ).

The main purpose of this paper is to explore the possibility of determining moving trend in a series of sub-times (10-year, 20-year, 30-year and 40-year) within hydro-meteorological time series. This methodology is called the moving trend method (MTM). Alongside the overall (holistic) trend across all records, a set of sub-time trends are presented to recognize the dynamic development behavior of trends. In this way, one can appreciate how trends have developed throughout the practically entire recording with scientific commentary. Sub-times are considered as repetition periods (recurrence intervals) and accordingly probabilities are associated with a set of partial trend slopes. MTM application is made for annual hydro-meteorological records from Danube River discharges, New Jersey state-wise temperatures and Istanbul City precipitation values.

Following a brief literature review in the Section 1 , this article consists subsequent 5 sections covering theoretical and practical aspects of MTM analysis methodology. The importance of the traditional statistical moving average procedure is explained in Section 2 . In connection with this, Section 3 explains the theoretical characteristics of the proposed MTM approach. Section 4 includes three different hydrometeorology recording applications with figures, tables, implications, and inferences. Section 5 presents the pros and cons of the proposed MTM methodology. Finally, Section 6 provides conclusive information about the limitations of the MTM approach and its useful aspects for further research.

2 Moving Average Significance

A series of arithmetic averages with a fixed number of data in a time series is called the moving average procedure, which moves through the series adjacent to the consecutive average. Prior to formal pattern identification in any hydro-meteorology time series, visual inspection reveals several short-term embedded features such as partial trends. Generally, within short periods, a time series has a few statistical and probabilistic components, including possible cascade of spikes (jumps), trend, periodicity, short- and long-term serial dependencies, and random variations. Numerous articles focus on holistic trend determination methodology such as the Mann-Kendal trend test (Mann 1955 ; Sen 1968 ; Kendall 1973 ), over a long-term systematic variation that is a linear function and shows the general trend.

The moving average method provides a series of consecutive averages over a period of m for which the number of data is smaller than whole data length, n. The moving average over m period represents the time series change as a horizontal line. There are several problems with the moving average procedure, such as determining the extension of the moving average to eliminate the original fluctuations in the time series. The resulting moving average time series cannot be used for future trend prediction, which is the main target for objective trend determination. Another problem is with horizontal lines, which can be replaced by straight lines that linearly increase or decrease linearly as partial and local trends; this is the main purpose of this paper recommendation as moving trend methodology (MTM).

3 Moving Trend Method (MTM)

It is well-known that global warming refers to the increasing temperature trend prevailing throughout the Earth; the effects of climate in any hydro-meteorology record series are not global, but local and regional in the forms of either increasing or decreasing trends. Like spatial globalism and regionalism, temporally any given hydro-meteorological time series can have trends over the entire recording period or over desired sub-periods in a series. As mentioned earlier in the Section 1 , MTM is based on innovative trend analysis modified for sub-period trend identification along the following points.

Decide on the duration of a series of periods to determine the trend series record. It is adapted in this paper for 10-year, 20-year, 30-year and 40-year. The last two periods are in line with the World Meteorological Organization’s recommendation for climate change trend identification studies (WHO 2017 ). Meanwhile, the holistic trend for the entire record is also considered for comparison purposes,

Starting from the first record, the trend component of each period is determined according to the innovative trend analysis approach. For this purpose, the trend slope, S T , for each period is calculated according to the following expression.

where m is the duration of period, \({\overline{\text{X}}}_{\text{S}\text{H}}\) ( \({\overline{\text{X}}}_{\text{F}\text{H}}\) ) is the second (first) half of the original time series data during the period,

To plot the linear trend over the corresponding period, the crossing point is taken as the period mid-time, t mp , (abscissa), and the arithmetic mean value of the as \({\overline{\text{X}}}_{\text{P}}\) (ordinate) Thus, moving trend sequences are obtained within the hydro-meteorological data.

4 Application

The Danube River is the second longest river in Europe after the Volga in Russia, with a length of approximately 2,850 km (1,770 mi) and is a river in many parts of Europe, including Austria, Slovakia, Hungary, Croatia, Serbia, Romania, Bulgaria, Moldova and Ukraine. It has a record starting from 1840. Another very long temperature record exists from the USA state of New Jersey, starting from 1895. Rainfall records on the European side of the city of Istanbul in Turkey start from 1936. The application of the MTM is performed for different long-term annual hydro-meteorology variables, including records for Danube River discharges (1840–2010), New Jersey state-wise temperature (1895–2010) and Istanbul precipitation (1939–2020) with their statistical parameters in Table 1 .

For the Danube River annual discharge records, there are 4 graphs in Fig. 1 for the 10-year, 20-year, 30-year and 40-year sequences of moving trend components. The same graphs also show the overall holistic trend component of the records for comparison. The moving trend component of each period fluctuates around the holistic trend, which is an overall approach and does not give detailed information as the serial trend evolution across the whole records in shorter periods than the number of records. The following points are important in the interpretation of comparative trend study.

In all Graphs, Holistic Trend has a Slightly Increasing Slope of 79 m 3 /year,

In Fig. 1 a, there are two extremely wet periods in the past between 1842 and 1852 and 1872–1882, respectively in the 10-year consecutive periods. Since then all 10-year moving trends are in decreasing form and in continuous reduction and the most decreasing 20-year trend appeared between 1982 and 2002. This irrespective or increasing or decreasing tendencies. There are several increasing trends around the overall (holistic) trend an especially in the most recent 10-year (1992–2002) it is also very close to general tendency,

As for the 20-year moving trends, there was only one increasing trend in the past between 1842 and 1862 (see Fig. 1 b). Since then all consecutive 20-year moving trends are in decreasing form and their collective appearance is also in decrease until the end of record where the most decreasing moving trend slope is during 1982–2002 period,

Figure 1 c includes moving trends for 30-year periods, which is the World Meteorological Organization’s proposed standard duration for climate change studies (WHO 2017 ). The only increasing 30-year moving trend is between 1972 and 2002 period, all other moving trends have decreasing tendency around the holistic trend,

Danube River annual discharge trends for a 10-year, b 20-year, c 30-year, d 40-year

The common conclusion from all graphs is that the most dynamic water formations occurred around 1900. Each of these graphs has distinct moving average trends bouncing from increasing (wet) to the decreasing (dry). It is not possible to obtain interconnected moving trend series.

Each graph in Fig. 1 reveals the relative position of different duration MTM components relative to the holistic trend. It is obvious that there are increasing and decreasing finite-period trends above and below, which provide detailed dynamic changes to make the future finite-trend forecast of water resources planning, operation and management. The following points are important for this type of works.

In Fig. 1 a, the 10-year periods represent 10 increasing moving trends, of which 3 are above the overall trend, 5 are below. The number of decreasing trends is 5 and only 2 of them are above the overall trend. This information implies that increasing trends are more effective than decreasing alternatives throughout the entire recording period. Especially after 1980, increasing trends are observed. In the light of these explanations, it can be predicted that the 10-year moving averages of the annual discharges of the Danube River are bound to increase.

Comparing the 10-year moving trends with the 20-year trends in Fig. 1 b shows that the latter trends are closer to the general holistic trend, and most of the trends in this period tend to decrease with increasing tendency throughout the entire recording period,

The 30-year moving trends in Fig. 1 c are closer to the overall holistic trend than the 20-year duration trends. Although there have been two decreasing trends in the past 60 years, there is an increasing trend in the last 30 years,

In Fig. 1 d, the 40-year moving trends are almost like the 30-year trends, there are two trends that have increasing appearance in the last 80 years.

Figure 2 shows all moving trends from different periods collectively to make comparison of the most severe situations dynamically throughout the record length. Among all these periods 30-year moving trends attract attention because of the WHO ( 2017 ) report recommendation. The most variation domain of moving trends is attached with 10-year period. Although short period but has the biggest increasing and decreasing trend variation domain from 4400 m 3 /sec to 6400 m 3 /sec.

Danube River discharge records moving trend collection

As for the New Jersey annual temperature record MTM, the overall trend and moving trends are shown in Fig. 3 . There is a holistic trend that increases with a slope of 0.0175 o F/year. It is worth paying attention to the following points.

In Figs. 3 a and 10-year consecutive moving trends show continuously increasing and decreasing tendencies that take place around the holistic trend. The general tendency of moving trends follows overall temperature increase with ups and downs,

Figure 3 b shows the 20-year consecutive moving trends that follow the overall (holistic) increasing tendency. Especially, between 1970 and 2010 20-year trends are in the form of decreasing trends, but their positions show temperature increases,

Three 30-year periods moving trends are shown in Fig. 3 c, which have standard lengths of records for climate change impact interpretations as recommended by World Meteorological Organization. Increasing and decreasing moving trends are in good accord with the holistic trend. The middle point of each moving trend shown increase in the position of increasing and decreasing trends,

There are two 40-year moving trends in Fig. 3 d. Just opposite the holistic trend, 1930–1970 trend has ignorable slope, whereas the recent moving trend during 1970–2010 period has decreasing trend but its middle point position is higher than the previous one. This point indicates that whether increasing or decreasing 40-year trends they have increasing middle point location by time,

New Jersey annual temperature trends, a 10-year, b 20-year, c 30-year, d 40-year

In Fig. 4 all period moving average trends are shown collectively, which shows that whatever is the moving trend period there is in all increasing trend position irrespective of increasing or decreasing tendency. Although there appear decreasing moving trends in the most recent periods compared to the same set moving trend position is in increase.

New Jersey State wise temperature records moving trend collection

Annual precipitation records of Istanbul City from 1940 to 2012 are shown in Fig. 5 with moving trend components over each periods. The overall trend tends to increase with slope 0.0507 cm/year, which means a very small increase in precipitation record moving means.

For 10-year moving trends below the overall trend indicates possible water shortages that have occurred at various time durations in the past and it is clear from Fig. 5 a that decadal shortages are about 86% (see Fig. 5 a). The only decade with increasing precipitation seems from 1962 to 1972,

20-year moving trends during 1972–1992 and 1992–2002 periods are the same with the holistic trend that shows quite stationary rainfall regime on the average in the last 40 years of the record,

As for the 30-year period during 1982–2002 there appears a slight decrease around the holistic trend. The middle point position of this moving trend lies on the holistic trend line. This means that during the first (second) half of this period moving trend decrease is above (below) the holistic trend (see Fig. 5 c),

Figure 5 d is for 40-year period decreasing moving trend component, which has almost the same slope as the holistic trend again with half slightly over and the next half below the holistic trend.

Istanbul annual precipitation trends, a 10-year, b 20-year, c 30-year, d 40-year

Moving trends of different periods are shown collectively in Fig. 6 , where only 10-year period moving trends expose quite sharp increasing and decreasing trends. Especially, the last 30-year trend indicates a decreasing trend of which the middle point lies on the holistic trend with early (late) half is above (below) the holistic trend.

Istanbul/Florya precipitation records moving trend collection

A set of slope values for moving trends are given in Table 2 for each data set including different periods 10-year, 20-year, 30-year and 40-year, respectively. A wide variety of moving trend slope variations can be noticed for each period. The mean and the standard deviations of each period are also given in the same table.

Of course, the average values are based on positive (increasing) and negative (decreasing) trend values and thus generally show the general trend towards overall positive or negative moving trend values. A common point as for the 30-year period is concerned that all the hydro-meteorology records have decreasing trend slopes. For example, Istanbul moving trend average values are negative for all periods, indicating that it is possible to expect precipitation reduction in future time periods.

5 Discussion

There are different trend identification methodologies in the open literature and some of them require restrictive assumptions that are not very satisfactory for natural hydro-meteorological recording time series, for which the assumption of serial independence is most important. Some are interested in non-parametric procedures in which the natural order of hydro-meteorology time series is replaced, for example, by orders as ranks (Spearman 1904 ). Most of these trending tests require long time series as their results are in biased for short samples. In this article, innovative trend analysis (ITA) methodology is applied as a different approach to overall trend determination, because moving trend components are valid even for time series samples that are significantly shorter than the World Meteorology Organization recommendation of 30 years (WHO 2017 ). Overwhelmingly, theoretical trend identification or practical procedures predominantly consider the entire sample length of a given time series. Many are biased because they do not meet key constraining assumptions such as serial independence, normal (Gaussian) probability distribution function (PDF) or long sample lengths.

Comparing the results of MTM procedure for different sub-periods of a hydro-meteorology time series provides the dynamic behavior of finite length trend development over the recording period. Generally, moving trends fluctuate around the overall trend line. Comparing up and down moving trend numbers to understand the frequency of short-term percentages result in higher (lower) values than the overall trend. Instead of taking future action according to the overall full recording time trend, making dynamic comments considering the sequence of MTM trends, and accordingly, better trending probability in the future.

The innovative aspect of this paper is the dynamic description of the desired partial-time trend analysis instead of holistic classical trend trends that do not show cycles of increasing and decreasing trend evolution over the entire length of the record. Once possible finite-term trend slopes have been identified, it is possible to make same-term future trend slope forecasts for improved water resources management planning and operation.

The MTM methodology provides information about trend slopes, so that during which moving trend duration the maximum and minimum increasing and decreasing slopes occur and, if necessary, average slopes for these two types of trends can be determined.

6 Conclusions

As a result of the greenhouse gas (GHG) emissions into the troposphere, global warming has led to climate change at different increasing or decreasing rates in different parts of the world. In climate change impact studies, trend analysis procedures are important to decide whether there is an increasing or decreasing overall trend for a hydro-meteorological variable at a location, unlike general circulation (climate) models (GCMs). This paper provides moving trend analysis for several sub-periods in a given time series records and their comparison with classical holistic trend procedures in the literature. For this purpose, a dynamic trend evaluation study is carried out by investigating subsequent special moving trend for 10-year, 20-year, 30-year and 40-year periods. The application of the proposed moving trend methodology is given for annual records of Danube River discharges, New Jersey state-wise temperatures and Istanbul City precipitation. Moving trending also provides percentages of decreasing (increasing) trend activity over the entire time series. It is recommended that the application of moving trend analysis sheds light on more dynamic structural behavior of hydro-meteorological records. The only limitation for holistic trend analysis is at least 30 years of data, which is recommended by the World Meteorological Organization and is also valid for the moving trend analysis methodology. There has not been yet any other research using MTM analysis, but in future water resources management studies such part-time trends tend to provide important information about dry and wet trend tendencies and their likely duration.

Data Availability

Data can be provided upon request from the corresponding author.

Abbas M, Arshad M, Shahid MA (2023) Zoning of groundwater level using innovative trend analysis: case study at Rechna Doab, Pakistan. Water Resour Irrig Manage - WRIM 12:64–80. https://doi.org/10.19149/wrim.v12i1-3.3155

Article   Google Scholar  

Achite M, Ceribasi G, Ceyhunlu AI et al (2021) The innovative polygon trend analysis (IPTA) as a simple qualitative method to detect changes in Environment—Example detecting trends of the total monthly precipitation in semiarid area. Sustainability 13:12674. https://doi.org/10.3390/su132212674

Alashan S (2018) An improved version of innovative trend analyses. Arab J Geosci 11(3):50

Ashraf MS, Ahmad I, Khan NM, Zhang F, Bilal A, Guo J (2021) Streamflow variations in monthly, seasonal, annual and extreme values using Mann-Kendall, Spearmen’s rho and innovative trend analysis. Water Resour Manag 35:243–261

Birpınar ME, Kızılöz B, Şişman E (2023) Classic trend analysis methods’ paradoxical results and innovative trend analysis methodology with percentile ranges. Theor Appl Climatol. https://doi.org/10.1007/s00704-023-04449-6

Box GEP, Jenkins GM (1976) Time series analysis, forecasting and control. Holden-Day, San Francisco

Google Scholar  

Burn DH, Hag Elnur MA (2002) Detection of hydrologic trends and variability. J Hydrol 255:107–122

Cailas MD, Cavadias G, Gehr R (1986) Application of a non-parametric approach for monitoring and detecting trends in water quality data of the St. Lawrence River. Water Pollut Res J Can 21(2):153–167

Article   CAS   Google Scholar  

Demaree GR, Nicolis C (1990) Onset of Sahelian drought viewed as a fluctuation-induced transition. Q J R Meteorol Soc 116:221–238

Douglas EM, Vogel R, Kroll CN (2000) Trends in floods and low flows in the United States: impact of spatial correlation. J Hydrol 240:90–105

Esit M, Kumar S, Pandey A et al (2021) Seasonal to multi-year soil moisture drought forecasting. Clim Atmospheric Sci 4:1–8. https://doi.org/10.1038/s41612-021-00172-z

Fanta SS, Yesuf MB, Saeed S et al (2022) Analysis of precipitation and temperature trends under the impact of climate change over ten districts of Jimma Zone, Ethiopia. Earth Syst Environ. https://doi.org/10.1007/s41748-022-00322-0

Gan TY (1998) Hydro-climatic trends and possible climatic warming in the Canadian prairies. Water Resour Res 34(11):3009–3015

Güçlü YS (2018) Multiple Şen-innovative trend analyses and partial Mann-Kendall test. J Hydrol Elsevier 566:685–704

Hamed KH (2008) Trend detection in hydrologic data: the Mann–Kendall trend test under the scaling hypothesis. J Hydrol 349:350–363

Hamilton JP, Whitelaw GS, Fenech A (2001) Mean annual temperature and annual precipitation trends at Canadian biosphere reserves. Environ Monit Assess 67:239–275

Hipel KW, McLeod AI, Weiler RR (1988) Data analysis of water quality time series in Lake Erie, Water Resour. Bull 24(3):533–544

CAS   Google Scholar  

Hirca T, Eryılmaz TG, Niazkar M (2022) Applications of innovative polygonal trend analyses to precipitation series of Eastern Black Sea Basin, Turkey. Theor Appl Climatol 147:651–667. https://doi.org/10.1007/s00704-021-03837-0

Hirsch RM, Slack JR (1984) A nonparametric trend test for seasonal data with serial dependence. Water Resour Res 20(1):727–732

Hirsch RM, Slack JR, Smith RA (1982) Techniques of trend analysis for monthly water quality analysis. Water Resource Res 18(1):107–121

IPCC (2007) Climate Change 2007: impacts, adaptation, and vulnerability. Contribution of Working Group II to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge

Kalra A, Piechota TC, Davies R, Tootle GA (2008) Changes in U.S. streamflow and western U.S. snowpack. J Hydrol Eng 13:156–163

Kendall MG (1970) Rank correlation methods, 4th edn. Griffin, London

Kendall MG (1973) Rank correlation methods. Charles Griffin Book Series, London 

Lee H, Apley DW (2011) Improved design of robust exponentially weighted moving average control charts for autocorrelated processes. Qual Reliab Eng Int 27(3):337–352

Lins HF, Slack JR (1999) Streamflow trends in the United States. Geophys Res Lett 26(2):227–230

Mann HB (1945) Nonparametric tests against trend. Econometrica 13(3):245–259

Mann HB (1955) Non-parametric tests against trend Econometrica. J Econom Soc 13(3):245–259

Mateus C, Potito A (2022) Long-term trends in daily extreme air temperature indices in Ireland from 1885 to 2018. Weather Clim Extrem 36:100464

Milly PCD, Betancourt J, Falkenmark M, Hirsch RM, Kundzewicz ZW, Lettenmaier DP, Stouffer RJ (2008) Stationarity is dead: whither water management? Science 319:573–574

O’Brien NL, Burn DH, Annable WK, Thompson PJ (2021) Trend detection in the presence of positive and negative serial correlation: a comparison of block maxima and peaks-over-threshold data. Water Resour Res 57:e2020WR028886

Pastagia J, Mehta D (2022) Application of innovative trend analysis on rainfall time series over Rajsamand district of Rajasthan state. Water Supply 22:7189–7196. https://doi.org/10.2166/ws.2022.276

Sen PK (1968) Estimates of the regression coefficient based on Kendall’s tau. J Am Stat Assoc 63(324):1379–1389

Şen Z (2012) Innovative trend analysis methodology. J Hydrol Eng 17(9):1042–1046

Şen Z (2017) Hydrological trend analysis with innovative and over-whitening procedures. Hydrol Sci J 62(2):294–305

Sonali P, Kumar DN (2013) Review of trend detection methods and their application to detect temperature changes in India. J Hydrol 476:212–227

Spearman C (1904) The proof and measurement of association between two things. Am J Psychol 15:72–101. https://doi.org/10.2307/1412159

Taylor CH, Loftis JC (1989) Testing for trend in lake and groundwater quality time series. Water Resour Bull 25(4):715–726

Ullah I, Ma X, Yin J, Omer A, Habtemicheal BA, Saleem F, Iyakaremye V, Syed S, Arshad M, Liu M (2022) Spatiotemporal characteristics of meteorological drought variability and trends (1981–2020) over South Asia and the associated large-scale circulation patterns. Clim Dyn 60:2261–2284

Van Belle VG, Hughes JP (1984) Nonparametric tests for trends in water quality. Water Resour Res 20(1):127–136

WHO (2017) WMO Guidelines on the Calculation of Climate Normals. WMO-No. 1203. p 18

Xie Y, Liu S, Huang S, Fang H, Ding M, Huang C, Shen T (2022) Local trend analysis method of hydrological time series based on piecewise linear representation and hypothesis test. J Clean Prod 339:130695

Yeh AB, Lin DK, Zhou H, Venkataramani CH (2003) A multivariate exponentially weighted moving average control chart for monitoring process 52 variability. J Applied Statistics 30(5):507–536

Yu YS, Zou S, Whittemore D (1993) Non-parametric trend analysis of water quality data of river in Kansas. J Hydrol 150:61–80

Yue S, Wang CY (2002) Applicability of pre-whitening to eliminate the influence of serial correlation on the Mann-Kendall test. Water Resour Res 38. https://doi.org/10.1029/2001WR000861

Download references

Open access funding provided by the Scientific and Technological Research Council of Türkiye (TÜBİTAK). The research received no funds.

Author information

Authors and affiliations.

Faculty of Engineering and Natural Sciences, İstanbul Medipol University, Beykoz, 84810, Turkey

You can also search for this author in PubMed   Google Scholar

Contributions

All parts are contributed by the single author.

Corresponding author

Correspondence to Zekâi Şen .

Ethics declarations

Ethical approval.

The manuscript is conducted in the ethical manner advised by the targeted journal.

Consent to Participate

Not applicable.

Consent to Publish

The research is scientifically consented to be published.

Conflict of Interest

The author declares no conflict of interest to any party.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Şen, Z. Moving Trend Analysis Methodology for Hydro-meteorology Time Series Dynamic Assessment. Water Resour Manage (2024). https://doi.org/10.1007/s11269-024-03872-2

Download citation

Received : 01 April 2024

Accepted : 23 April 2024

Published : 15 May 2024

DOI : https://doi.org/10.1007/s11269-024-03872-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Hydro-meteorology
• Find a journal
• Publish with us
• Track your research