Addressing many of the world’s contemporary challenges requires a multifaceted and integrated approach, and interdisciplinary research (IDR) has become increasingly central to both academic interest and government science policies. Although higher interdisciplinarity is then often assumed to be associated with higher research impact, there has been little solid scientific evidence supporting this assumption. Here, we provide verifiable evidence that interdisciplinarity is statistically significantly and positively associated with research impact by focusing on highly cited paper clusters known as the research fronts (RFs). Interdisciplinarity is uniquely operationalised as the effective number of distinct disciplines involved in the RF, computed from the relative abundance of disciplines and the affinity between disciplines, where all natural sciences are classified into eight disciplines. The result of a multiple regression analysis (n = 2,560) showed that an increase by one in the effective number of disciplines was associated with an approximately 20% increase in the research impact, which was defined as a field-normalised citation-based measure. A new visualisation technique was then applied to identify the research areas in which high-impact IDR is underway and to investigate its evolution over time and across disciplines. Collectively, this work establishes a new framework for understanding the nature and dynamism of IDR in relation to existing disciplines and its relevance to science policymaking.
Many of the world’s contemporary challenges are inherently complex and cannot be addressed or resolved by any single discipline, requiring a multifaceted and integrated approach across disciplines (Gibbons et al., 1994; Frodeman et al., 2010; Aldrich, 2014; Ledford, 2015). Given the widespread recognition today that cross-disciplinary communication and collaboration are necessary to not only pursue a curiosity-driven quest for fundamental knowledge but also address complex socioeconomic issues, interdisciplinary research (IDR) has become increasingly central to both academic interest and government science policies (Jacobs and Frickel, 2009; Roco et al., 2013; NRC, 2014; Allmendinger, 2015; Van Noorden, 2015; Davé et al., 2016b; Wernli and Darbellay, 2016). Accordingly, various national and international programmes, focusing especially on promoting IDR, have recently been launched and developed in many countries through specialised research funding and grants or through staff allocations (e.g., Davé et al., 2016a; Gleed and Marchant, 2016; Kuroki and Ukawa, 2017; NSF, 2019).
Driving these pro-IDR policies and the attendant rhetoric is an implicit assumption that IDR is inherently beneficial and has a more substantial impact compared with traditional disciplinary research. However, this assumption has rarely been supported by solid scientific evidence, and in most cases, the supposed merit of IDR has been based on anecdotal evidence from specific narrative examples or case studies (for related perspectives, see e.g., Jacobs and Frickel, 2009, p. 60; Weingart, 2010, p. 12). Considering the fact that significant resources have been and are being invested in promoting IDR, better clarity regarding the relationship between interdisciplinarity and its potential benefit, particularly the research performance, could help increase accountability for such policy actions.
Extant literature has investigated the relationship between interdisciplinarity and the research performance by using various data sources and methodologies, with different operationalisation of both dimensions (e.g., Steele and Stier, 2000; Rinia et al., 2001; Rinia et al., 2002; Adams et al., 2007; Levitt and Thelwall, 2008; Larivière and Gingras, 2010; Chen et al., 2015; Elsevier, 2015; Yegros-Yegros et al., 2015; Leahey et al., 2017). Owing to such diverse investigation approaches, it is unsurprising that the results are usually neither consistent nor conformable and sometimes are even contradictory among the literature. Given this situation, it is desirable that a more robust and reproducible methodology be developed and implemented to systematically assess the value of IDR in practice. The present study seeks to contribute to this goal by developing a new testbed for IDR evaluation. The focus is especially placed on highly cited paper clusters known as the research fronts (RFs), which are defined by a co-citation clustering method (Small, 1973). In this new approach, the research interdisciplinarity is characterised by the disciplinary diversity of the papers that compose the RF, and the research performance is operationalised and measured as a field-normalised citation-based measure at the RF level.
This proposed RF-based approach has three major advantages over common approaches that focus, for instance, on individual papers (Steele and Stier, 2000; Adams et al., 2007; Larivière and Gingras, 2010; Chen et al., 2015; Elsevier, 2015; Yegros-Yegros et al., 2015) to investigate the potential effect of interdisciplinarity on high-impact research. First, through the analyses of RFs, it is possible to capture a snapshot of the most lively, animated and high-impact research currently being undertaken in the academic sphere, since the papers composing RFs are classified as the most highly cited papers for each science discipline. As science policymakers, leaders, funders and practitioners are often most interested in promoting and supporting high-impact research, the evidence and insights obtained through this investigation of RFs can assist them in formulating more accountable policy recommendations that otherwise cannot be adequately addressed. Second, the RF is a unique manifestation of knowledge integration from different science disciplines. By construction, the interdisciplinarity operationalised at the RF level does not represent a mere parallel existence of discrete knowledge sources from multiple disciplines; rather, it indicates the state of the knowledge integration from multiple disciplines to create new knowledge syntheses. This organic scientific knowledge structure can be captured more effectively and robustly through RFs than through, for instance, an individual paper’s reference list. Consequently, the emergence of a new high-impact research area will also be more reliably detected at the RF level than at the paper level. The third advantage of the proposed RF-based approach is related to the technicalities. As discussed, RFs are unique self-organised units of knowledge in which bibliographically important information is effectively compressed and integrated. As this study considers thousands of papers, it is considerably more efficient and effective to handle RFs compared with a multitude of papers while conducting data retrieval, analysis and visualisation. These multifold advantages of the RF-based approach enable this study to comprehensively and uniquely assess the value of interdisciplinarity.
The analyses in this study were based on the data retrieved from the Essential Science Indicators (ESI) database, published by Clarivate Analytics, and data published by the National Institute of Science and Technology Policy (NISTEP) of Japan. In this section, the definitions for the main terms used in this paper—the RFs, the research areas, the research impact and the interdisciplinarity index—are provided. Subsequently, the regression model specification used in this study and the rationale behind it are detailed.
The bibliometric data for the research papers (regular scientific articles and review articles) and citation counts were derived from more than 10,000 journals indexed in the Web of Science Core Collection published by Clarivate Analytics. The master journal list is updated regularly, with each journal being assigned to only one of the 22 ESI research areas (see Supplementary Table S1). Given a pre-set co-citation threshold, the original ‘ESI-RFs’ were defined based on the number of times the pairs of papers had been co-cited by the specified year and month within a five-year to six-year period. The ESI-RF investigation in this paper was focused on papers classified as ‘Highly Cited Papers’ in the ESI database, which are the top 1% for annual citation counts in each of the 22 ESI research areas based on the 10 most recent publication years.
Based on the ESI framework, the NISTEP’s Science Map dataset (NISTEP, 2014, 2016, 2018) defines a set of ‘aggregate RFs’ using a second-stage clustering in each of the three data periods: 2007–2012, 2009–2014 and 2011–2016, which are denoted in this study as S2012, S2014 and S2016, respectively. Each dataset comprised approximately 800–900 of such ‘aggregate RFs’ (hereinafter referred to as ‘RFs’). The i-th RF in the aggregate dataset S = S2012 ∪ S2014 ∪ S2016 was denoted by RFi. After excluding two RFs with missing data, there were |S| = 2,560 RFs collected for the total data period (2007–2016), with a cumulative number of 53,885 papers (Table 1).
The interdisciplinarity index (2) is unique because it is conceptualised as the effective number of distinct disciplines involved in each RF and is robust regarding the research discipline classification scheme. Specifically, it has the special property of remaining invariant under an arbitrary grouping of the constituent disciplines, given that the between-discipline affinity is properly defined for all pairs of disciplines. For instance, suppose one is interested in measuring the interdisciplinarity of RFi based on the classification scheme \(<\mathscr
Using this formula, the interdisciplinarity index for each RF in S was obtained, from which it was found that 43.6% of the RFs were mono-disciplinary (i.e., Δ = 1) and more than half were interdisciplinary (Fig. 2a; median = 1.2, range = 2.5; see also Supplementary Fig. S1a).
Based on the aforementioned operationalisations of the research impact and the interdisciplinarity index, the relationship between the two variables was analysed using a regression analysis method. As the histogram analysis showed that the original research impact distribution was skewed, it was log-transformed so that the distribution curve was closer to a normal curve (Fig. 2b; mean = 1.2, SD = 0.83; see also Supplementary Fig. S1b). The scatterplot of the log-transformed research impact against the interdisciplinarity index indicated that these variables were relatively linearly related (Fig. 2c; see also Supplementary Fig. S2a–c). Subsequently, the following multiple linear regression model was investigated:
$$\ln \left(where, xi was a l × k vector for predictive variables, and β was a k × l vector for the regression coefficients, which were the unknown parameters to be estimated (with k being some integer). To deal with the possible issue of heteroscedasticity, the model was analysed using heteroscedasticity-robust standard errors (i.e., the Huber-White estimators of variance). In addition, a test for serial correlation (i.e., the Breusch-Godfrey Lagrange multiplier test) was conducted as a post-estimation procedure, which indicated that there was no serial correlation between the residuals in each model considered (see below).
For comparability, five different regression models corresponding to different specifications of the predictive variables were analysed and labelled Models 1–5, with the following sets of predictive variables, respectively, defined for each model:
In Model 1, the interdisciplinarity index was used as the only predictive variable, which was added to the intercept term (constant). In Model 2, the variables associated with IntlCollab and IntlCiting, denoting the proportion of internationally collaborated papers in papers comprising an RF and in the citing papers, respectively, were included as additional predictive variables. Models 3, 4 and 5, in the same manner, represented the prior model with a new set of predictive variables, respectively, added as follows: Year dummy variables for the different years (2012, 2014 and 2016) of the Science Map to capture the possible time-fixed effects; a ‘Research Area’ control set to represent the proportion of papers belonging to each research area A ∈ \(<\mathscr
In interpreting the regression results, each regression coefficient βk (i.e., the k-th component of β in Eq. (3)) indicated that a one point increase in the predictive variable xk was associated with βk point increase in ln(I), or equivalently, [exp(βk)−1] × 100% increase in the research impact (I) at the specified significance level. Care should be taken in interpreting the results for the proportion variables (IntlCollab, IntlCiting, ‘Research Area’ and ‘Country’ control sets) as the regression coefficients for each of these variables represented the effect on the criterion variable (i.e., the log-transformed research impact) associated with a 100% increase in the proportion variable. For the time-fixed effects, the base category was chosen as Year = 2014, against which the effects of the other two data periods (corresponding to Year = 2012 and 2016) were measured. For the ‘Research Area’ control set, the effect of the proportion of each research area in \(<\mathscr
The results of the multiple regression analyses for all the five models (n = 2,560; two-tailed) are summarised in Supplementary Table S4. Based on the adjusted-R 2 for each model (the bottom row of the table), Model 5 was found to be the preferred model in terms of the goodness-of-fit, and therefore, this model was considered in detail in this study; see Table 2 for the summary table.
Moreover, each RF’s concrete disciplinary composition was indicated by the direction(s) towards which the RF’s peak tails (see Supplementary Fig. S4). For instance, in the Science Landscape for 2009–2014 (Fig. 3b), there is a high research impact peak (I = 100.7) near the centre that has one tail towards ‘Comp & Math’ and another tail towards ‘Basic Life Sciences’ (the solid square region). In light of the original NISTEP’s Science Map dataset (NISTEP, 2016), this peak corresponds to the RF characterised by feature words such as ‘RNA Seq’ and ‘next generation sequencing’. Then, intuitively, this correspondence indicates that during this period, there was a scientific breakthrough related to new sequencing technology that occurred at the intersection of these two disciplines. Further technical and mathematical details including the explicit functional form of the 3D research impact profile are presented in Supplementary Methods and Discussion.
Provided the above encoding, the Science Landscape diagrams (Fig. 3a–c) clearly illustrate how the shape of interdisciplinarity has changed over the three data periods. It is noticeable that the overall landscape of the research impact has never been static, monolithic nor homogeneous; rather, it evolves dynamically, both over time and across disciplines. One of the most remarkable features can be seen in the northwest of the map (dashed circle region) at the low ivory-white-coloured ‘mountains’ in 2007–2012 (Fig. 3a), where new high-impact RFs are evolving and developing into a group of yellow-coloured mid-height ‘mountains’ in the years up to 2009–2014 (Fig. 3b) and towards 2001–2016 (Fig. 3c). This dynamic research impact growth indicates the increased IDR focus around the region during the data period. Thus, this visualisation can assist identifying where the scientific community’s focus of attention is undergoing a massive change, where high-impact IDR is underway worldwide, and where new knowledge domains are being created. Each landscape appears to represent the superposition of the following two research impact evolutionary patterns; one that has steady, stable or predictable development that accounts for the ‘global’ or ‘evergreen’ structure of the landscape, and the other that represents a breakthrough in science or a discontinuous innovation, induced ‘locally’ in a rather abrupt or unpredictable manner. The challenge of science policy, therefore, is developing ways to address each of these dynamic evolutionary patterns and the mechanism thereof and to promote IDR in a more evidence-based manner with increased accountability for the investments made.
This study revisited the classic question as to the degree of influence interdisciplinarity has on research performance by focusing on the highly cited paper clusters known as the RFs. The RF-based approach developed in this paper had several advantages over more traditional approaches based on a paper-level or journal-level analysis. The multifold advantages included: quality-screening, cross-disciplinary knowledge syntheses, structural robustness and effective data handling. Based on data collected from 2,560 RFs from all natural science disciplines that had been published from 2007 to 2016, the potential effect of interdisciplinarity on the research impact was evaluated using a regression analysis. It was found that an increase by one in the effective number of distinct disciplines involved in an RF was statistically highly significantly associated with an approximately 20% increase in the research impact, defined as a field-normalised citation-based measure. These findings provide verifiable evidence for the merits of IDR, shedding new light on the value and impact of crossing disciplinary borders. Further, a new visualisation technique—the Science Landscape—was applied to identify the research areas in which high-impact IDR is underway and to investigate its evolution over time and across disciplines. Collectively, this study established a new framework for understanding the nature and dynamism of IDR in relation to existing disciplines and its relevance to science policymaking.
The new conceptual and methodological framework developed to reveal the nature of IDR in this paper would be of interest to a wide range of communities and people involved in research activities. However, as with any bibliometric research, this study also faced various limitations that may have impacted the general validity of the findings, and thus, its practicability in the real policymaking process is necessarily limited. To conclude, some of these key issues and challenges are highlighted.
First, both the regression analysis results and the Science Landscape visualisations should be assessed with caution as they may be highly dependent on the research area classification scheme, which is not unique. Research area specifications other than those used in this study could also have been applied. For instance, a factor-analytical approach (Leydesdorff and Rafols, 2009) to identify a ‘better justified’ set of academic disciplines could be useful in providing a more nuanced assessment and understanding of the nature of interdisciplinarity and could possibly have higher robustness and reliability. Moreover, a different research area arrangement along the edge of the circular map would have resulted in different Science Landscape visualisations, and the cross-disciplinary spectrum of research impact might have been more plentiful or profound than observed in this study.
Second, in relation to the first point, the quantification of the affinity between the research areas could have been refined in other acceptable ways. Our rationale behind the definition of the between-discipline affinity based on the Jaccard-index was that papers from closer (i.e., with higher affinity) research areas were more likely to be co-cited, and thus more likely to belong to the same ESI-RF (see Supplementary Methods and Discussion). In this approach, the affinity matrix was defined solely using the bibliometric method, and therefore its matrix elements may have been more or less biased because of the publication/citation practices of the existing disciplines. Consequently, it may have failed to capture the inherent ‘true’ between-discipline affinities responsible for the ‘true’ interdisciplinarity operationalised at the RF level.
Third, it is unlikely that the regression model specification used in this study included every salient research impact predictor. For example, factors such as the types of research institute, departmental affiliations, individual journal characteristics and funding opportunities (e.g., funding agencies and programmes/fellowships) were not considered in the model owing to their unavailability in the dataset. Moreover, the links between the different scientific specialties irrespective of their academic discipline could have also influenced the research performances. These omitted variables may also have affected the regression results because they may be associated with both the criterion variable (i.e., the research impact) and some predictive variables including the interdisciplinarity index.
Finally, there are inherent limitations in using citation-based methods to evaluate research performance. Combining bibliometric approaches with expert judgements from qualitative perspectives will be favoured to extract the policy implications and recommendations from a wider context. Although the societal impacts of research (see e.g., Bornmann, 2013) were beyond the scope of the present work, it is hoped that this study’s findings can be extended to incorporate such societal aspects. In so doing, it is also important to consider not only the benefits but also the costs of IDR (Yegros-Yegros et al., 2015; Leahey et al., 2017) for interdisciplinary approaches to provide viable policy options for decision-makers.
With further conceptual and methodological improvements, it is hoped that future studies can reveal more about the nature of IDR and its intrinsic academic and/or societal value by overcoming some of the aforementioned limitations. Continued efforts will contribute to the development of the more evidence-based and accountable IDR strategies that will be imperative for addressing, coping with and overcoming contemporary and future challenges of the world.
The datasets generated and/or analysed during this study are not currently publicly available, but are available from the corresponding author on reasonable request.
This work was conducted as part of the in-house research activities of the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan. This work also contributes to the MEXT’s ‘Science for RE-designing Science, Technology and Innovation Policy (SciREX)’ programme, hosted at the National Graduate Institute for Policy Studies (GRIPS), for which the author serves as Policy Liaison Officer. The views and conclusions contained herein are those of the author and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the government of Japan.
