Sisyphus random walk

In this paper we consider a particular version of the random walk with restarts: random reset events which bring suddenly the system to the starting value. We analyze its relevant statistical properties like the transition probability and show how an equilibrium state appears. Formulas for the first-passage time, high-water marks and other extreme statistics are also derived: we consider counting problems associated naturally to the system. Finally we indicate feasible generalizations useful for interpreting different physical effects.


I. INTRODUCTION
The Sisyphus random walk , the stochastic process to be introduced in this paper, is an infinite Markov chain whose dynamics can be expressed as follows: at every clock tick the process can move rightward (or upward) one step, with a certain given probability, or else return to the initial state, from where it is restarted. Such apparent simplicity is misleading as this simple evolution law can exhibit a surprisingly complex and rich behavior The process considered here bears some analogy with the punishment, in the Greek mythology, of Sisyphus, the first king of Ephyra, who was sentenced to climb up a hill carrying a heavy, slippery boulder and watch it roll down again to the starting point in an endless cycle. Remarkably, it also underlies the behavior of some physical mechanisms. In the context of Doppler laser cooling, "Sisyphus effect" is a well known mechanism by which alkali atoms in the presence of a light field raise from the ground Zeeman level to higher excited states or sublevels. This up hill climbing process increases the probability to be optically pumped into a minimum potentialenergy state from where the process restarts. In addition, it involves a loss of momentum so after each Sisyphus cycle, the total energy of the atom decreases by a certain amount. 1 In a different setting, such system may be used as an idealized model of the random dynamics of a "mobile" in a trap, say, who is trying to climb stepwise a ladder or wall given that at every step there is a common probability of slipping to the bottom, resulting in the need to restart again. In such situation the distribution of the time to escape the trap becomes a natural question.
The hallmark of such processes is the possibility to display return-to-the-origin behavior, a common feature in real life systems. The seminal work of Manrubia and Zanette [2] considering Markov chains where a reset mechanism operates has motivated new interest in the field and presently the dynamics of systems with resets is being subjected to intense study. In [3,4] Brownian motion with resets was considered while in [5] resets were incorporated to a compound Poisson process with constant drift. Such intermittent strategies have been considered in general mathematical frameworks [3][4][5][6][7][8], but also in more specific contexts, like behavioral ecology where the browsing activity of living organisms (as, e.g., capuchin monkeys) may be suddenly interrupted to return to a preferred location [9,10], or econophysics where, modifying Gibrat's law to include reset events, has made it possible to account for the power law's distribution of firm's growth [11]. The present paper continues this line of research and considers a different special case of stochastic processes with a reset mechanism. See [12][13][14][15][16] for other developments in this regard.
The paper is organized as follows. In Sec. II we introduce the process at hand and remind some basic concepts of renewal theory. In Sec. III we derive an explicit expression for the propagator , the transition probability function of the process, an essential magnitude for understanding the dynamics that provides a paradigmatic example of the use of renewal concepts. The statistics analysis of several extreme events as, e.g., the first-passage time or the maximum of the process, and related survival probabilities, is discussed in Secs. IV and V where we also analyze in detail related counting problems and recurrence issues. In Sec. VI we generalize the model to include different physical effects. By allowing the reset probability to be random or site-dependent interesting generalizations are obtained. The resulting random walk may be used to model physical systems which become "increasingly anxious" to restart as they drift far away from the initial state, as happens in situations as diverse as Sisyphus cooling or formation -and eventual collapse-of built-up structures like stalagmites. Depending on the election for the reset probability we find that the equilibrium state has heavy or light tails and obeys a geometric, a zeta or a Poisson distribution. Other possible generalizations include the interesting possibility of having a random walk on the integers which upon reset may drift upward or downward with different probabilities. Here the problem of finding the optimal reset strategy appears naturally. Conclusions are drawn in Sec. VII and, finally, we complete some technical details in the appendix.

II. SISYPHUS RANDOM WALK AS A RENEWAL PROCESS
The Sisyphus random walk, X t , is an infinite Markov chain on the positive integers -namely X t ∈ {0, 1, 2, . . .} for t ∈ {0, 1, 2, . . .}-whose one-step evolution can be expressed as follows: If at time t the walker is at a given location, X t = ℓ, then at time t + 1 one has that is, at every clock tick the process can increase one unit, or return to the ground state, from where the evolution continues. If the initial condition is set in such a way that X 0 = 0, as we do consider, this return to the ground state can be understood as a restart of the process. Typical sample paths of the process X t are (irregular) sawtooth functions, namely, piecewise linear functions that slope upward and then sharply fall at reset times, see Fig. 1. Equation (1) stresses the fact that the elements q ℓ of the transition matrix between different locations on the chain can depend on the present state ℓ of the system, an assumption consistent with the Markov property. We rule out, however, the possibility that the transition matrix depends on the chronological time t, or on some other hidden variable. Under such hypothesis, the process is time homogeneous.
An appropriate dependence of q ℓ on ℓ may befit a model where, as a result of learning abilities -or some exogenous circumstance-the system becomes more (or less) anxious to return to the origin the farther off it is. In the rest of the paper we study the most relevant statistical magnitudes of X t corresponding to the case q ℓ = q, a constant parameter. This simplification reduces the algebraic intricacy of a model which, despite this, is still mathematically rich and complex. We indicate some results corresponding to different ℓ-dependence in Sec. VI.
To this end we note that, under such assumption, the existence of resets gives rise to an underlying renewal structure 2 and hence, for most statistics of interest, renewal-type equations can be employed to advantage over conventional random walk theory, see [18]. The reader is also referred to the interesting paper [10], where some properties of a related discrete-time random walk -which evolves via Lévy flights on the line combined with resets-are considered.
A key quantity in renewal theory is the renewal function, m(t; t 0 ), the mean number of reset events in a given interval (t 0 , t]. Most of the formulas employed in the paper consider different properties of the walker in some future instant t, subject to the knowledge of its state at time t 0 , t 0 < t. Due to the time homogeneity of the process, those expressions actually depend not on the calendar time but on the time lapsed between the two events, τ ≡ t − t 0 , and thus we define m(τ ) ≡ m(t 0 + τ ; t 0 ).
The renewal function m(τ ) satisfies a renewal equation, which reflects the two possible scenarios that appear depending on the relative value of τ with respect to τ * , the random instant at which the first reset (after t 0 ) takes place. Using that P {τ * = k} = q k−1 (1 − q) and P {τ * > τ } = q τ , (2) where P{· · · } is the probability relative to the set {· · · } one finds that m(τ ) must solve Equations like (3) can be conveniently analyzed by recourse to the so called z-transform: where z is a complex variable. Equation (3) is solved in terms of this object to find where E[·] denotes the expectation of its argument.

III. TRANSITION PROBABILITY
We start our analysis of X t with the determination of the transition probability of the process, the propagator : The propagator gives the probability of finding the walker in position ℓ at the future instant t, if we known that it is in ℓ 0 at the present time t 0 < t. Note that, as discussed previously, it depends on time only via τ = t − t 0 but must depend on both ℓ and ℓ 0 : Indeed we do not expect translation invariance since the reset mechanism favours a particular point, the origin; as a consequence the probability function for the walker position does not contain all physical information and the full transition probability, Eq. (8) below, is required to describe the dynamics. The equation that p(ℓ, τ ; ℓ 0 ) satisfies follows by analyzing the different situations that may present depending on whether t 0 + τ * , namely the first reset after t 0 , occurs or not in the interval (t 0 , t 0 + τ ]. If δ k,k ′ is the Kronecker delta, the transition probability becomes p(ℓ, τ − τ * ; 0) in the first case and δ τ,ℓ−ℓ0 in the latter -see also [14] for a similar reasoning in the context of Brownian motion with resets to the origin. In view of all this, and the probabilities (2), p(ℓ, τ ; ℓ 0 ) must satisfy the following equation: Letting first ℓ 0 = 0 in (6) an equation for p(ℓ, τ ; 0) follows which can be handled by using z-transforms. We find where Θ (u) is the (right-continuous) Heaviside step function: Θ (u) = 1 for u ≥ 0, and Θ (u) = 0 for u < 0. Hence, having started from the origin the position has (truncated) geometric distribution p(ℓ, τ ; 0), which decreases with ℓ. However, if q > 1/2, the mode of the distribution is found at ℓ = τ , as Fig. 2 shows. By insertion of (7) in Eq. (6) we obtain the complete propagator as Hence the only effect of having started from ℓ > 0 is to shift the highest accesible site from τ to τ + ℓ 0 , maintaining the probabilities of the remaining accessible states {0, 1, . . . , τ − 1} unchanged: and p(ℓ 0 + τ, τ ; ℓ 0 ) = p(τ, τ ; 0). From Eq. (8) the conditional mean position is given by and is a monotone function of τ going from ℓ 0 to the limit value q/(1 − q), see Fig. 3.   3 show the sample path, distribution and, respectively, the conditional mean of the processes X t when q = 4/5 (and ℓ 0 = 2). In this case the average time between resets is 5 and the average limit value is 4 -see Eqs.
(2) and (10) above. Notice how the mean function fails to capture the sharp drops in trajectories at reset times.
From Eq. (8) we obtain the stationary state a geometric distribution with mean µ = q/(1 − q) and typical standard deviation σ = √ q/(1 − q). Thus the system is recurrent and ergodic and, given enough time, attains an equilibrium distribution. This ergodicity could be expected on physical grounds since the incorporation of this reset mechanism guarantees that the system will not be driven too far off from the origin. Note that this stationary state X ∞ ≡ lim τ →∞ X τ has the same distribution as X τ * −1 , the distance covered before the motion restarts: cf. Eqs. (2) and (11). We stress that those remarkable concurrences in Eqs. (9) and (12) do not extend to general election of q ℓ .

IV. EXTREME-TIME STATISTICS
In the next two sections we study some statistical properties of several extreme functionals associated to the process X t . In our context, an extreme event is simply a physical observable or quantity related to X t that has attained a minimum or a maximum. It turns out that there are several of them which are quite interesting from a physical viewpoint.
A. First-passage statistics Let ℓ be a given level. The first-passage time, or hitting time [6], F (ℓ), represents the minimum lapse of time needed for the process to travel from the ground state to the given site, or for the walker to exit the trap by climbing an elevation ℓ. We begin considering the analysis of the first-passage time probability of the process to this level, P(t, ℓ), namely, the probability that the process which is initially at X 0 = 0 reaches level ℓ for the first time at instant t. The renewal equation for P(ℓ, t) reads where the first term accounts for the eventuality that the process reaches ℓ without restarting, and the summation contains those cases in which the first reset takes place at time t * = k ≤ ℓ. (We remark that related techniques have been used elsewhere in the context of reset systems; note, in particular, the close similarity with the derivation of the survival probability in [10].) Taking the z-transform of (15) with respect to the time variable t one gets P(z, ℓ) = (qz) The value of any given P(t, ℓ) could be obtained from the t-order derivative of P(z, ℓ) at z = 0, , and leads to the following formula, valid for t ≥ ℓ + 1, see App. A: where ⌊·⌋ denotes the floor function, In Fig. 4, we plot the corresponding distribution probability for ℓ = 10 as a function of t. It is fortunate that despite the unwieldiness of expression Eq. (17) most statistical magnitudes can be obtained in an easy way. In particular, the mean first-passage time can be easily obtained from P(z, ℓ): and hence the mean time to reach ℓ grows exponentially with the distance [3].

B. Record statistics
By construction, the first time our Sisyphus random walk visits a given threshold ℓ, F (ℓ), it scores a record : the highest value reached by the process up to that time.
Since the process is right-continuous, one can guarantee that X t ′ < ℓ for 0 ≤ t ′ < F (ℓ), and X t = ℓ, at t = F (ℓ).
The next question that naturally arises within this context is what are the properties of the interval spent between two records of the process [19], the inter-record statistics? To answer this question we are going to generalize (13) and define F (ℓ; t 0 , ℓ 0 ), the first time the process reaches ℓ after time t 0 , and call P(t, ℓ; t 0 , ℓ 0 ) its associated probability When ℓ > ℓ 0 , as we assume along in this section, P(t, ℓ; t 0 , ℓ 0 ) is the probability that at time t, t > t 0 , the process achieves the new local maximum ℓ, since X t ′ < ℓ, for t 0 ≤ t ′ < F (ℓ; t 0 , ℓ 0 ). We will clarify later on the connection between P(t, ℓ; t 0 , ℓ 0 ) and R(t, ℓ; t 0 , ℓ 0 ), the probability that the process sets a new record ℓ at time t, provided it scored record ℓ 0 at time t 0 . As in Sec. III, the time-homogeneity of the process implies that P(t, ℓ; t 0 , ℓ 0 ) is a function of t and t 0 through the time interval τ = t−t 0 , P(τ, ℓ; ℓ 0 ) ≡ P(t−t 0 , ℓ; 0, ℓ 0 ). In this case, the equation for P(τ, ℓ; ℓ 0 ) reads with the first term representing the contingency in which the process increases steadily during τ consecutive steps, passing from ℓ 0 to ℓ, whereas the summation contains those cases where the first reset happens after a lapse of τ * = k ≤ ℓ − ℓ 0 . Since the restart takes the process to the origin, in the sum appears P(τ, ℓ) = P(τ, ℓ; 0), see Eq. (14).
Basic moments of R(τ, ℓ; ℓ 0 ) follow from Eq. (22) by taking derivatives at z = 1. In particular, the mean inter-record lapse is Finally, note that we have not demanded that the two records are consecutive, that is, that there is no additional record in the time interval. In our case, however, the statistics associated to this contingency can be easily derived from R(τ, ℓ; ℓ 0 ), by merely setting ℓ = ℓ 0 + 1, R(τ, ℓ) ≡ R(τ, ℓ; ℓ − 1). Thus, for instance, C. Mean recurrence time In this section we will finish the analysis of P(τ, ℓ; ℓ 0 ) when ℓ ≤ ℓ 0 . In this case, the system can only attain ℓ having first been reset to the origin, and therefore whose generating function is given by where we have used again (16). Equation (26) can be inverted by the methods described above. In particular, as expected on intuitive grounds, the probability that the chain ever visits ℓ, starting at ℓ 0 ≥ ℓ, is P(z = 1, ℓ; ℓ 0 ) = 1, and all states are positive recurrent. Actually, site ℓ is visited infinitely often with probability 1.
The mean recurrence time satisfies as the classical ergodic theorem predicts [18]. Further, the time average of the process is given by .

A. Number of sites visited
Another interesting magnitude, closely connected to P(t, ℓ), is the high-water mark: the highest value ℓ that the process has reached for a fixed time t, namely This quantity marks the threshold between those sites that have been already reached, from those that have not. Classical extreme-value theory is devoted to study the distribution of this statistics, typically assuming strong conditions on the increments of the process, i.e., independence and identically distributed, conditions that are not met here. Note that 0 ≤ M t ≤ t and, in contrast to  Consider the probability associated to M t , H(ℓ, t), Obviously, can only happen if the process suffers just one restart either at t = 1 or at t − 1. For general values, 0 ≤ ℓ ≤ t − 2, the derivation of the corresponding probabilities is not straightforward. The key fact is that Hence and therefore it follows We can see a practical example in Fig. 6, where for t = 30 and two different choices for q, q = 4/5 and q = 9/10, we plot H(ℓ, t) versus ℓ. In the lower panel, observe the local maximum at ℓ = t, and the kink at ℓ = t/2. Exact determination of the mode of the distribution is not an easy task.

B. Number of visits
Given a fixed time t, the high-water mark level counts the number of sites that have been visited. A related counting physical observable involves N (t; ℓ), the number of visits to a fixed level ℓ up to time t, and N (n, t; ℓ), the probability that this number equals n ∈ N, i.e., N (n, t; ℓ) ≡ P {N (t; ℓ) = n} . (32) In the extreme case t = ℓ, obviously, with probability 1. In addition, the probability that the site ℓ have never been visited is just However, to go beyond these general statements and determine N (n, t; ℓ) with all generality is far from trivial. We resort again to renewal arguments to obtain a set of equations with hierarchical structure: 3 In this formula we can identify three different kinds of contributions. The single term assumes that no reset has taken place up to time t, and the process visits just once each of the levels between 0 and ℓ. The first summation contains those cases for which the first reset takes place before the walker visits the targeted level ℓ. In the second summation we find those cases in which the process has reached or passed ℓ once by the time of the first restart.
The z-transform of Eq. (34) with respect to the time variable t leads to for n ≥ 1, while the value of N (0, z; ℓ) can be readily obtained from Eqs. (16), (30), and (33), Hence, for n ≥ 1, the z-transform is given in an explicit way as Fortunately, there is no need to invert this expression to obtain the mean and main moments of the distribution. Indeed, say The "mean occupation number", i.e., the mean number of visits to level ℓ at time t ≥ ℓ, follows by inversion of this result as and depends linearly on t − ℓ. In particular, letting ℓ = t, ℓ = 0, or t → ∞, the results at the beginning of the section are recovered.

VI. ALTERNATIVE DYNAMICS
In this section we relax some of the assumptions made in the core of the main text, to indicate the capabilities and possible generalizations of Sisyphus random walk.

A. Random probabilities
The first variation to the previous setup is obtained assuming that q ℓ is still independent of ℓ, but is random, and hence not a fixed parameter. This situation corresponds to a walker climbing a ladder whose slip probability does not change with the step but remains unknown, due to insufficient information on the walker idiosyncrasy. By replacing q by Q, a random variable, some results appropriate to this case follow from the previous expressions, by taking expectations with respect to Q. Concretely, unconditional probabilities may be derived this way. However, conditioning gives information which may partially pin down the slip probability of the walker.
For instance, suppose that α > 0, i.e., 1−Q has a Pareto distribution on the interval [0, 1]. Then one has that where Γ (x) is the Gamma function. This means that inter-reset times follow a Zipf-Simon-Yule (or discretized Pareto) distribution, well known in certain areas of econophysics, like wealth distribution. Recall that Eq. (41) is used to model the frequency of words in languages, or the size of objects randomly chosen from certain types of populations, see [20], pp. 260 and following.
The mean value of the inter-reset time now reads and is finite if and only if α > 1. Note that this excludes the uniform distribution, α = 1. Mean first-passage times or mean inter-record times are always unbounded magnitudes for ℓ ≥ 1. The same law governs the properties of marginal and equilibrium probabilities of the process, cf. Eqs. (7) and (11), Thus, the equilibrium distribution p(ℓ) has heavy tails with Pareto exponent α + 1.

B. Shrinking probabilities
In this section we drop the requirement that q ℓ be constant [4]. In this case most previous results, including the distribution of τ * must be generalized appropriately. We focus on a system whose return probability increases with the location of the walker. This assumption is adequate to describe, say, the process of formation and eventual collapse of stalagmites or a house of cards. A natural election is where q 0 , 0 < q 0 ≤ 1 is a constant parameter that accounts for the probability of leaving the ground level and α > 0. To show how the methodology must be deployed, we detail the simple case corresponding to α = 1, namely q ℓ = q 0 ℓ + 1 . Figure 7 shows how typical trajectories under this dynamics are less likely to access higher values than the initial model with q ℓ = q 0 , cf. Fig. 1. It follows that the reset probability satisfies This leads to a mean inter-reset time E [τ * ] = e q0 , and to the following renewal equation for p(ℓ, τ ; 0), cf. Eq. (6) with ℓ = 0, Appropriate use of the z-transform permits solving this equation as Here we have introduced the exponential sum function e n (−q 0 ), defined as the n-th Taylor polynomial for the exponential function: Letting τ → ∞ in Eq. (46) we find that the stationary state has Poisson distribution with parameter q 0 and mean position 1/q 0 .
The results can be generalized to an arbitrary election for q ℓ . Skipping the details, we note that for τ ≥ ℓ (48) The renewal theorem implies then that as long as one has E [τ * ] < ∞, the stationary distribution is Thus, both the finite-time position distribution and stationary state follow in closed form given the distribution of renewals P {τ * > ℓ} = q 0 · · · q ℓ−1 and the function m(τ ) -which can be recovered solving Eq. (3) appropriately generalized, cf. [17].

C. Sisyphus random walk on the integers
Here we consider the Sisyphus random walk generalized to a case where trajectories could, after every reset , either increase or decrease linearly with probabilities ρ and 1 − ρ, respectively. Concretely, given X t = 0 we generalize Eq. (1) to while if X t = 0 then X t+1 = ℓ + sgn(ℓ), with probability q, 0, with probability (1 − q).

VII. CONCLUSIONS
We have analyzed an extremely simple random walk with constant deterministic dynamics that may randomly return to the origin and have determined its main statistics; in particular, the first-passage time and high-water marks of the process are discussed. It turns out that this simple evolution law is misleading and the corresponding dynamics is surprisingly complex and exhibits a rich behaviour. Nevertheless suitable renewal ideas may be used to simplify the analysis. We suggest generalizations of the system, appropriate to different physical settings; in particular a situation where the tendency to return increases with the distance to the initial state can be incorporated into our formalism. The resulting system is then capable to provide a gross description of Sisyphus cooling, and formation and growth of houses of cards and icicles. Depending on the election, Zipf, geometric or Poisson distributions are found to describe the equilibrium state.
One has G(z) = 1 Hence and therefore, collecting the terms with z τ , we have for τ ≥ ℓ + 1, and where ⌊x⌋ = max {k ∈ Z | k ≤ x}.