Dynamic Criteria: a Longitudinal Analysis of Professional Basketball Players' Outcomes

This paper describes the fluctuations of temporal criteria dynamics in the context of professional sport. Specifically, we try to verify the underlying deterministic patterns in the outcomes of professional basketball players. We use a longitudinal approach based on the analysis of the outcomes of 94 basketball players over ten years, covering practically players' entire career development. Time series were analyzed with techniques derived from nonlinear dynamical systems theory. These techniques analyze the underlying patterns in outcomes without previous shape assumptions (linear or nonlinear). These techniques are capable of detecting an intermediate situation between randomness and determinism, called chaos. So they are very useful for the study of dynamic criteria in organizations. We have found most players (88.30%) have a deterministic pattern in their outcomes, and most cases are chaotic (81.92%). Players with chaotic patterns have higher outcomes than players with linear patterns. Moreover, players with power forward and center positions achieve better results than other players. The high number of chaotic patterns found suggests caution when appraising individual outcomes, when coaches try to find the appropriate combination of players to design a competitive team, and other personnel decisions. Management efforts must be made to assume this uncertainty. En este artículo describimos las fluctuaciones en el tiempo del rendimiento de jugadores profesionales de baloncesto buscando patrones deterministas y de qué tipo son. Para ello, analizamos los resultados de 94 jugadores profesionales mediante un estudio longitudinal de series temporales de diez años de duración. Analizamos las series temporales utilizando las técnicas que se proponen desde la teoría de sistemas dinámicos no lineales. Mediante estas técnicas podemos descubrir los patrones subyacentes de los resultados sin tener que realizar asunciones previas sobre la linealidad o no linealidad de los datos, ni transformaciones de los mismos para que se ajusten a priori a una distribución. En los resultados encontrados, la mayoría de los jugadores muestran un patrón determinista (88.30%), de los cuales la mayoría son caóticos (81.92%) que obtienen mejores resultados que los lineales. El alto número de patrones caóticos encontrados parece indicar que debemos ser precavidos a la hora de evaluar y tomar decisiones sobre el rendimiento de los jugadores, y que la gestión de equipos debe asumir que la incertidumbre es una parte importante en este contexto.

This paper focuses on the description of temporal dynamics of criteria in the context of professional sport. As such, it is of interest for dynamic criteria literature and for the applied perspective also, as it provides information about the possibility of predicting individual outcomes over time.

Theoretical framework: dynamic criteria and nonlinear dynamical systems theory
A criterion could be defined as "a sample of performance (including behavior and outcomes), measured directly or indirectly, perceived to be of value to organizational constituencies for facilitating decisions about predictors or programs" (Austin & Villanova, 1992, p. 838). Dynamic criteria are defined by Deadrick and Madigan (1990) as "systematic changes in critical job behaviors or outcomes over time that are attributable to individual differences" (p. 719). This implies that criteria change over time, but these changes could be totally or partially deterministic. Therefore, some authors (Chan, 2005;Deadrick, Bennett, & Russell, 1997;Hofmann et al., 1993) place specific emphasis on time and longitudinal designs as essential elements in the study of dynamic criteria research.
Dynamic criteria have generated studies that aim to determine the temporal consistency of criteria (e.g., Hanges et al., 1990;Landis, 2001;Zickar & Slaughter, 1999), the stability of predictor-criteria relationships (e.g., Deadrick & Madigan, 1990;Ployhart & Hakel, 1998;Zyphur, Bradley, Landis, & Thoresen, 2007), the relationship between temporal variables and criteria (e.g., Hofmann, Jacobs, & Gerras,1992;Sturman, 2003), and the effect of dynamic criteria on assessments (e.g., Reb & Cropanzano, 2007;Reb & Greguras, 2010). Literature review of empirical dynamic criteria, which is summarized in Table 1, reveals some outstanding points. The most important is the lack of consensus about what constitute an adequate trend for describing the fluctuations in criteria. This is due to the fact that the ultimate sources of variability are within-subject (Deadrick & Madigan, 1990), and individuals have their own and characteristic fluctuations in criterion (Zyphur et al., 2007). Other interesting conclusions in the literature review are: a) researchers tend to use objective measures to study criteria; b) in most cases, these measures are performance outcomes; and c) these studies have been conducted over a wide range of occupations and a variety of sample sizes.
Taking into account the above, linear or near-to-linear models are the most frequently used (Beal, Weiss, Barros, & MacDermid, 2005). This seems to reflect the exclusive use of techniques which are not capable of capturing other possible distributions. Consequently, more effort is needed to describe the criteria fluctuations, such as analyzing data with broad spectrum techniques. For this purpose, we prefer to use techniques that are not based on any previous assumption about data due to loss of potentially valuable information (Sturman, Cheramie, & Cashen, 2005).
Consistent with Reb and Greguras (2008) and Fisher (2008), we consider deviations from the mean as part of the criteria, not necessarily errors or biases. We are interested in assessing the underlying pattern these distributions follow. We may find nonlinear patterns, but this is an exciting challenge for research. Despite being enthusiastically received by the scientific community, nonlinear relationships have proved to be difficult to examine (Molenaar, 2004), so we need data, methods, and tools that are sensitive to capture changes. Fortunately, the complexity theory, subsequently called after the nonlinear dynamical systems (NDS) theory (Guastello, Koopman, & Pincus, 2009), allows a better understanding of organizational behavior (Maguire, McKelvey, Mirabeau, & Öztas, 2006;Pastor & García-Izquierdo, 2007). In fact, empirical research in some Industrial and Organizational Psychology topics has reached promising results, such as those about flow (Ceja & Navarro, 2009;, coordination and team effectiveness (Guastello & Guastello, 1998), or work motivation (Navarro & Arrieta, 2010).
The NDS theory is based on the study of dynamic systems by time series (Heath, 2000). A time series is based on a collection of sequential observations made over time (Chatfield, 1996), which form patterns, defined as "a reoccurring sequence or continuum of events with time as the parameter" (Schroeck, 1994, p. 73). Using the techniques derived from NDS, we can distinguish between deterministic and non-deterministic patterns (Nowak & Vallacher, 1998). When a pattern is described as deterministic, it is assumed that the future evolution of the pattern can be predicted from the knowledge of its current state (Nowak, Lewenstein, & Vallacher, 1994). Deterministic patterns can be further divided into linear patterns, which maintain cause-effect proportionality, and nonlinear ones, which do not. Otherwise, a pattern is described as non-deterministic when the future situation of the pattern is not related to its previous evolution.
One notable advantage of NDS techniques is their capability to detect a particular kind of nonlinear patterns: chaotic patterns. Chaos refers to an intermediate situation between randomness and determinism (Wiggins, 1988), a dynamic with aperiodic and restricted values, which is sensitive to and depends on initial conditions (Mathews, White, & Long, 1999). Chaotic patterns are easily confusable with random patterns, but in the first case, a short-term prediction is possible, whereas in the second case, no type of prediction is possible.
In this study, we are interested in individual's variability over time, and in developing and implementing studies about intraindividual variation (Molenaar, 2004). We use NDS, which has not been used in dynamic criteria research to date, to analyze the outcomes of professional basketball players without any previous assumption about their patterns across time.

Research context: professional basketball
According to Barnes and Morgeson (2007, p. 266), basketball can be described, as "a sport in which a team of 12 players competes against another team. At any given time, only 5 members are actively participating in the competition, though other team members may be substituted in at any time [...] each team has the opportunity to attempt to both score points (i.e., accumulate points for their team by having the player put the ball through a hoop) and prevent the other team from scoring points." Dynamic systems are characterized by the interaction between their elements and their evolution over time (Nowak & Vallacher, 1998). Consequently, we should consider basketball inherently as a dynamic system. Each player assumes a specific role in the team in a context characterized by high task interdependence (Landis, 2001) and uncertainty, where inputs, processes, or outputs of a work system lack predictability (Wall, Cordery, & Clegg, 2002).
Moreover, O*NET classifies basketball under Job Zone two (some preparation needed) with Specific Vocational Preparation (SVP) between four and six, that is, from three months to one year. As a result, we have considered basketball as a low-to-medium cognitive-demanding job. These characteristics (interaction, interconnected elements, high task interdependence, uncertainty) support the use of the NDS approach in our research.
In addition, this research context is especially adequate for our aims due to some methodological reasons we shall explain. Firstly, individual outcomes of each player were precisely measured and registered at every match during a ten-year follow up study, which allowed us to obtain robust time series. Analyses focused on each individual performance at each time period, so fluctuations in the same individuals can be observed, and we can determine the curve fit to the data. Secondly, these are public data published on the official website, so other researchers can replicate our analyses with the same pool of data. Thirdly, they are objective data, so they are not affected by subjective artifacts. Further, from a career development point of view, a professional sportsman's career is completed in few years (Alfermann & Stambulova, 2007). This feature increases the value of the longitudinal design in this context, as a ten-year study could be considered a professional sportsman's entire career development (Côté, 1999). Finally, some other previous studies have dealt with nonlinear dynamics (NDS) in sport settings (Abbott, Button, Pepping, & Collins, 2005;Chow et al., 2006), sport performance (Hardy & Partfit, 1991;Hardy, Partfit, & Pates, 1994;Hardy, Jones, & Gould, 1996;Hardy, Beattie, & Woodman, 2007), and work performance and effectiveness (Guastello,1995;Guastello & Guastello, 1998). All of this led us to think this context is an excellent framework for our investigation.

The present study
As mentioned above, basketball is characterized by interdependence and uncertainty. When uncertainty is high, as is usual in every match where team composition changes frequently, work roles must emerge dynamically in response to changing conditions and demands (Katz & Kahn, 1978). Following Pettigrew (1990), changes are to be explained more by loops than by lines, as causation of changes is not linear. We can conclude that each player's outcome is a result emerging from the dynamic interaction between each team player and those of the opposite team, that is, the outcome of feedback cycles. Thus, individual outcomes are also derived from the influence of coworkers and rivals. In fact, the coworkers are in a mixed-motive situation, as suggested by studies about cooperation and competition (Stewart & Nandkeolyar, 2007), where one may simultaneously cooperate with team members and promote oneself. Consequently, as there is high task interdependence (Landis, 2001) and uncertainty (Wall et al., 2002) in a dynamic system, we expect low stability in outcomes of basketball players: Hypothesis 1: outcomes of professional basketball players will present low stability across time.
Furthermore, we wished to determine whether the dynamics follow a deterministic pattern, a reoccurring sequence over time, whereas a non-deterministic pattern would take place when the future situation of the pattern is not related to its previous evolution. Dynamic criteria are assumed to fluctuate over time (e.g., Stewart & Nandkeolyar, 2007;Thoresen et al., 2004;Zyphur et al., 2007), showing a great variability of nonlinear shapes in medium-and high-demanding jobs (Sturman, 2003). Despite these fluctuations, individual outcomes change over time systematically (nonrandomly) (Sturman et al., 2005). Hence, we present our second hypothesis.

Hypothesis 2: the outcomes of professional basketball players follow a deterministic pattern.
Besides the deterministic pattern, NDS suggests that the most frequent behavior pattern across time is deterministic chaos (Vallacher & Nowak, 1997). Some empirical studies in organizational contexts support this (e.g., Ceja & Navarro, 2009;Cheng & Van de Ven, 1996;Guastello & Guastello, 1998;Navarro & Arrieta, 2010). In consequence, we present our third hypothesis: GARCÍA-IZQUIERDO, RAMOS-VILLAGRASA, AND NAVARRO 1136 DYNAMIC CRITERIA OF PROFESSIONAL BASKETBALL PLAYERS Hypothesis 3: the deterministic pattern found in the dynamics of the outcomes will be chaotic.
Summing up, these hypotheses will allow us to verify fluctuations in the outcomes in the selected sample, and whether there is an underlying deterministic pattern, and which pattern it is. Consequently, if the hypotheses are fulfilled, new questions in the theory and practice about the nature of dynamic criteria should be developed.

Sample
The sample comprises professional players of the Spanish Premier Basketball League. Data was collected between September and November of 2006 from the database of the Spanish National Basketball Association and they cover the results of each player throughout the 1996-2006 period. Cases that did not have at least 100 points were excluded from the study, as required when using techniques such as recurrence plots and Lyapunov exponents (Heath, 2000). Moreover, participants who had played more than 20% of games without recording were also eliminated, as the analysis techniques are sensitive to the lack of records in the series (Kreindler & Lumsden, 2007). Thus, the number of valid time series of the entire sample was reduced from 112 to 94.
All the participants were males. The number of matches played by each participant ranged between 115 and 436, with a mean of 256.91 (Mdn = 260.60, SD = 88.82). Table  2 shows the sample description. Mean age was 30.98 years (SD = 3.87 years, modes = 26 and 32) and the mean number of years of experience was 12.16 (SD = 3.51 years, mode = 14). Most of the participants played in the small forward position (30.85%), and the remaining positions were distributed as follows: 21.29% playmaker, center 18.08%, power forward 17.02%, and guard 12.77%.

Measures
We used diverse variables grouped in a composite criterion in order to explain most of the covariances of the true scores (Campbell, Blake, & Oswald, 1996). Out composite criterion, Stats (S), is an objective measure developed by the Spanish National Basketball Association based on the information about the outcomes per game for each player, a broadly accepted index in this setting. We followed the indication of Barnes and Morgeson (2007) for adjusting the scoring measures for time played (what they called "opportunity to play") in order to perform adequate comparisons. The final algorithm was: Where S is the composite criterion Stat, a is the number of points per game, b is the number of rebounds obtained per game, c is the number of assists per game, d is the 1137 number of steals per game, e is the number of personal fouls received per game, f is the number of blocked shots per game, w is the number of missed shots per game, x is the number of turnovers per game, y is the number of rebounds failed per game, z is the number of personal fouls committed per game, and t is minutes played per game.

Procedure and Analysis
The time series of S for each player was obtained. The series were refined to eliminate the games with missing data (the player had not been summoned or he had not played). We subsequently performed descriptive analyses (M, SD, and range). After this, we performed analyses to distinguish clearly between the different patterns of time series, in the line we have exposed in Ramos-Villagrasa and García-Izquierdo (2011). Our step-by-step analysis procedure is shown in Figure 1.
We began time series analysis with a preliminary visual exploration of the series. For this purpose, we plotted the observations against time (time plot) and the correlogram (a graph representing a set of autocorrelation coefficients of a time series). Visual inspection with these techniques is helpful to describe time series and should be conducted before attempting more complicated procedures (Chatfield, 1996). If a preliminary check shows that time series are mainly linear or nearly linear, a traditional approach is recommended (e.g., ARIMA models, hierarchical linear models); otherwise, the use of NDS is recommended.
The NDS approach offers three analyses to describe dynamics: recurrence plots, maximum of Lyapunov exponent, and surrogate data testing. As we have pointed out above, analyses from the complexity theory are not based on the assumption of any concrete structure of time series. The following analyses and software were employed in the present paper: a) the maximum Lyapunov exponent, with Chaos Data Analyzer (CDA) statistical program; b) recurrence plot, with Visual Recurrence Analysis (VRA 4.7); and lastly, c) surrogate data testing, with Time Series Analysis (TISEAN 3.0.1). As these analyses have hardly been used in dynamic criteria research, we shall describe each one (for further explanations, see Heath, 2000).
The maximum Lyapunov exponent is a quantitative indicator of time series patterns. It shows how variables become less predictable over time due to sensitivity to initial conditions. In the case of a linear pattern, the maximum Lyapunvov exponent is zero or less; otherwise, it could be chaotic or random. Nevertheless, we emphasize that Lyapunov exponents tend to overestimate chaos because they do are not sensitive enough to detect random patterns. In the case of randomness, Lyapunovs become very large, so actually they do indicate probably randomness. In those cases, chaos and randomness become much less distinguishable. In any event, finding a pure linear case in a time series is unlikely, increasing the risk of accepting the case as nonlinear when is linear. We controlled for this possibility by using recurrence plots.
A recurrence plot provides a graphical representation of the orbits of the phase space of a system and is one of the best ways to distinguish between nonlinear and random dynamics. It allows one to determine whether a one-dimensional system is random, chaotic, or linear. According to Heath (2000), the graphic representation is a rectangular chart made so that both axes represent the recorded number of the series, and with coordinates that correspond to the locations of the data values in the graph. In a recurrence plot, the pairs of points in the time series are compared. If the distance between a pair of points is less than an established distance, then a point is drawn, and if not, the space is left empty. Lines of identity are easily recognized by the upward marked diagonals called "recurrences" that characterize a chaotic time series. In a linear time series, the recurrence plot shows two thick lines with no recurrences. Lastly, in a random system, where any result is possible, many points appear across the chart, and no recurrences can be found. The main flaw of recurrence plots is that the interpretation is based on the researchers' expertise. These analyses (maximum Lyapunov exponents and recurrence plots) permit an approximation to the patterns of time series, although with some discrepancy between the two techniques. To clarify possible discrepancies, we used surrogate data testing.
Surrogate data testing is used to verify the randomness of time series. Starting from the original series, a random series is obtained, which maintains the same mean, variance, and structure of auto-correlation as the original series, but removing nonlinear dependency (Kugiumtzis, 2002). Many methods to perform surrogate data exists (e.g. Efron, 1982;Theiler, Lindsay, & Rubin, 1994;Schreiber & Schmitz, 1996). We choose the procedure detailed by Schreiber and Schmitz (1996) because it was tested in conditions where series are contaminated by measurement noise as usually happens in real contexts, demonstrating their appropriateness (Schreiber & Schmitz, 1997).
This method, also known as Amplitude Adjusted Fourier (AAFT) surrogates is implemented in the TISEAN software (surrogates command). We described AAFT procedure following Dolan and Spano (2001): firstly, the original data is rescaled to a normal distribution generating a time series of Gaussian white noise and ordering it according to the ranking of the original data. Secondly, a Fourier-transformed surrogate of the rescaled data is constructed. Finally, surrogate is scaled to the distribution of the original data by ordering original data according to the ranking of the Fourier-transformed surrogate. The surrogate series generated with AAFT have the same power spectrum and distribution as the data and converge to the amplitudeadjusted phase-randomized surrogate series if the data are normally distributed.
GARCÍA-IZQUIERDO, RAMOS-VILLAGRASA, AND NAVARRO Once we have the surrogate series, we perform a test to compare the generated series with the original series. Schreiber and Schmitz (1996) recommend the Theiler, Eubank, Longtin, Galdrikian, and Farmer (1992) rank-order test, which select a residual probability α of a false rejection, corresponding to a level of significance (1− α) × 100%. In the case of a bilateral test, it would be (2/α)-1 series. In our case, we established a 95% level of significance by generating 39 surrogate series with AAFT for each original one. If the test revealed no differences between the predictions from the original series and those from the surrogate series, it was concluded that the time series was random, avoiding Type I and Type II errors in pattern identification.
Summing up, results of preliminary visual explorations served to verify the first hypothesis of the study. Subsequently, NDS techniques allowed us to verify the second and third hypotheses. Lastly, we searched for differences based on time series patterns and player position using analysis of variance (ANOVA) and Scheffé's post hoc test to determine, for exploratory purposes, whether the pattern was related to the player's position.

Results
The study had 94 ranged time series. Descriptive statistics of Stats series are M = .35, SD = .42, and range between 140 and 382 registers.
Once preponderance of nonlinear patterns in time series is confirmed, the next step is to study the time series to determine the kind of dynamic of professional basketball players' outcomes (linear, chaotic, or random). We present these results step-by-step in Table 3. Firstly, the time plot showed high fluctuations in most of the time series (92.55%), and only seven (7.45%) were near to linearity. Correlogram exploration confirmed these results: the seven near-to-linearity time series exhibited correlograms similar to Chatfield's (1996) description of short-term correlations (correlations tend to become successively smaller), and the rest were near-to-random series (without correlations, or isolated and disorganized correlations). Based on the results of the preliminary visual inspection, our first hypothesis was supported, and we followed the NDS techniques to identify the patterns of time series. Table 3 (steps 2 to 4) shows the differences in the number of patterns depending on the test used, caused by the limitations of both techniques. Consequently, surrogate data testing, which served as a test contrast, was used to achieve the final results and better interpretation. In order to illustrate the numerical results, we present some graphic examples. Figure 2 shows some examples of the time plot (left side), recurrence plot (right side), maximum Lyapunov exponent, and surrogate data testing for each pattern (linear, chaotic, and random).
After maximum Lyapunov exponent and recurrence plot analyses, we perform surrogate data. After surrogate GARCÍA-IZQUIERDO, RAMOS-VILLAGRASA, AND NAVARRO 1140 ote. = 94 a ANOVA shows significant differences (F 2,91 ; p ≤ .001). Based on Scheffé follow-up test, differences were significant (99.00%) between: linear vs .chaos, and linear vs. random. data testing, we can definitely state that 81.92% of the players displayed chaotic dynamics, 6.38% linear dynamics, and 11.70% random dynamics. Deterministic patterns (linear and chaotic) were more frequent (88.30%) than random patterns (11.70%). These data partially confirm our second and third hypotheses. In the ANOVAs, significant differences were found in Stats between the linear pattern and the other patterns (random pattern: T = -4.24, p ≤ .01, chaotic pattern: T = -3.39, p ≤ .05): players with linear patterns had significant lower Stats than the other players. As random players achieved a higher Stats (.41) than chaotic players (.35), mean difference analysis was carried out, but no significant differences were found. Table 4 shows the mean Stats and patterns by player's position. Power forward players had the highest mean Stats (M = .46), and guard players the lowest (M = .29). We found significant differences between power forward players (with playmaker: T = -3.95, p ≤ .001, with small forward: T = -3.96, p ≤ .001), and between guard: T = 4.51, p ≤ .01); and between playmaker and center (T= -4.24, p ≤ .01). Nonetheless, the chaotic pattern was the most frequent (81.90%).

Discussion and conclusions
The study presented in this paper deal with dynamic criteria. This research contributes to that issue presenting an original approach and reporting empirical findings about the dynamics of sports performers that have not been reported before nor analyzed in the same way.
Moreover, it reveals the need to explore new theoretical and methodological measuring alternatives. As suggested in the literature, we found a substantial variability in criteria (i.e., Stewart & Nandkeolyar, 2007;Sturman, 2003), but we also found that most of the participants (88.03%) follow a deterministic pattern in their outcomes. Furthermore, most cases are chaotic (81.92%), a pattern unexplored in dynamic criteria research until now. We agree with Nowak et al. (1994, p. 284) that the "identification of patterns of spontaneously occurring changes in a system often constitutes the first step toward understanding the system's dynamics." In this sense, the present study shows that outcomes may be much more complex than was thought until now, because they may follow diverse trajectories (linear and nonlinear) and may adopt complex patterns (i.e. chaotic dynamic). Therefore, the NDS approach seems to be a complementary viewpoint to deal with dynamic criteria, providing renewed theoretical points of view and research methodologies when linear or near-to-linear approaches are not sufficient.
The high number of chaotic cases we found in this study indicates that basketball players' outcomes may be unpredictable in the long term, although this does not exclude other short-term predictions. Moreover, random cases produced higher Stats than linear ones. Nonetheless, there is still a remarkable difference between chaotic and random cases: the predictability of deterministic chaotic cases is low and only probable in the short term, but in random cases, by definition, it does not take place. This also raises new and interesting issues, such as whether these results could be replicated in other samples and jobs, or whether there are different types of chaotic dynamics, and what differences there may be between them. Insofar, power forward and center players have obtained better results in Stats than other players. This could be due their opportunities in the matches or even due to the Stats criterion, which, despite some limitations (Cooper, Ruiz, & Sirvent, 2009), is still being used by the Spanish League in decision making tasks.
Another contribution is related to methodology. We used several infrequent but promising techniques with dynamic criteria to obtain contrasted and consistent results, so we consider its use very recommendable in future studies. The NDS has proved its capacity to reach relevant contributions in Industrial and Organizational Psychology, as seen in the literature review.
Our study also has some implications for research and professional practice. If one assumes that dynamic criteria behave chaotically over time, one should conclude that the strategies of other players would affect the strategies of a particular player somehow, increasing the uncertainty derived from interaction between the players of both teams. A recent study with rugby players shows that the proximity of rivals increases the use of evasive maneuvers and, as a consequence, the emergence of unexpected behaviors (Passos et al., 2011). Moreover, a player who played a poor game last week could obtain better results next week; even the same training program may have unequal effects on players' outcomes. Consequently, all of this makes it more difficult to find the appropriate combination of players to design a competitive team. In fact, dynamics results suggest caution when appraising individual outcomes. Some of the properties attributed to chaotic systems such as fluctuations, lack of proportional cause-effect correspondence, and feedback effects due to the interaction, are very real issues that coaches, as direct supervisors, must cope with. From the NDS point of view, these problems are inherent properties of systems, instead of being due to the lack of information about the variables involved or to the absence of reliable measurement instruments. This does not imply that further exploration of the relations between variables or the improvement of measurement instruments is unnecessary, but rather that fluctuations in individual outcomes are inherent characteristics. Nor does it mean that one should avoid intervention. Efforts in management must be made, but should accept the uncertainty generated by the system. All of this supports the arguments of Reb and colleagues (Reb & Cropanzano, 2007;Reb & Greguras, 2008 about the necessity of considering dynamics in evaluation to obtain a more complete and realistic perspective.

Limitations and future research
The study presented in this paper is unique from those previously reviewed and adds to dynamic criteria literature in a number of ways, such as (a) it is focused on the description of intraindividual fluctuations in the outcomes of professional basketball players without making any assumptions about the shape of the patterns, (b) it examines the outcomes over a substantial period, which could be considered a player's entire career development, and (c) it shows some techniques derived from NDS that could be useful for the study of dynamic criteria. However, the investigation presented herein has some limitations. Firstly is the degree of generalization of the results. This study was carried out exclusively with professional basketball players and it is advisable to replicate this study in other work settings. Nevertheless, other studies (Day, Sin & Chen, 2004;Hofmann et al., 1992;Landis, 2001;Stewart & Nandkeolyar, 2007) have also used samples of sportsmen and made relevant contributions to dynamic criteria framework. In addition, the final sample size (n = 94) is not as large as we would have liked, but this is common in dynamic criteria research (e.g., Hofmann et al., 1992;Landis, 2001;Rambo, Chomiak, & Price, 1983;Reb & Cropanzano, 2007;Stewart & Nandkeolyar, 2007;Zickar & Slaughter, 1999). Moreover, each individual was measured up to ten years, that is, the players' entire career development (Côté, 1999), thus offering a long-term perspective scarcely investigated till now in dynamic criteria research.
Our results may also be affected by the phenomenon of range restriction. Professional sportsmen have undergone many continuous selection processes from their youth, and so their job performance outcomes are variable. Consequently, the number of chaotic dynamics is probably overestimated by participants' similarities. In this sense, Guion (1998, p. 361) stated, "It is my guess that truly linear regression does not occur in nature unless we have restricted range, specifically restricted to the middle of a complete distribution. For a variety of reasons, I suspect that a better fit in very low level or very high level jobs would be an ogival or logistic curve." It seems that nonnormal distributions that are characteristics of nonlinear phenomena linear would fit better than a linear one. This issue should be studied in future research with different samples with greater variability and different contexts, as well.
However, we cannot account for individual differences such as ability, motivation, organizational tenure, or other relevant characteristics. These must be the subject of further research, but we remind readers that the purpose of this paper was to focus on dynamics in outcomes, not on the source of such dynamics.
As we used objective measures, the influence of environmental variables could play a role. Environmental constraints and opportunities created by others and the mixed motives to cooperate and compete (i.e. Stewart & Nandkeolyar, 2007) are considered an inherent part of the outcomes, so coaches have to deal with them. Moreover, from the NDS point of view, outcomes should be the emerging result of these interacting variables.
Another research recommendation is to determine why the players have specific patterns. In addition, future research should contemplate the analysis of the dynamics of group performance, which are as relevant at the individual analysis carried out in this study. This would allow us to further our knowledge about the relation between the performance of the individuals who make up a group and the performance of the group itself.
Lastly, with regard to outcomes, they were measured with an objective composite criterion. There is still some controversy about what type of criterion is the most appropriate (Austin & Villanova, 1992;Dunnette, 1963;Sackett & Lievens, 2008), particularly concerning subjective DYNAMIC CRITERIA OF PROFESSIONAL BASKETBALL PLAYERS versus objective measures. It seems that most dynamic criteria researchers prefer objective measures, and in fact, the reliability of objective measures is better for simple jobs than for jobs with high cognitive demand (Stewart & Nandkeolyar, 2007), but we recommend that further research take into account the use of objective and subjective measures, as did Deadrick and Madigan (1990).
Summing up, our paper analyses the dynamic nature of criteria, conducting a deep and extensive research using long time series with a scarcely used approach, the NDS theory. Our findings are similar to those of other relevant studies (e.g., Beal et al., 2005;Deadrick & Madigan, 1990;Hofmann et al., 1993;Stewart & Nandkeolyar, 2007), but the chaotic patterns found open new and refreshing questions about the topic. Thus, future investigations are needed in this and some other complementary directions. As Guion (1998, p. 361) stated, "what is needed is an expansion of the options and perspectives of modeling by inviting the development and testing of nonlinear models as well."