This paper presents the construction of an automaton termed the Concurrent Probabilistic Reversible Automata (CPRA). It models the distributed systems which exhibit probabilistic behaviour and also relies on backtracking as the basis to make a system fault tolerant. Here, the basic concepts of non-probabilistic automata have been extended by introducing the discrete probabilities on the set of transitions for modelling the probabilistic nature of computing environment. In addition, concurrency has been implemented by defining parallel joint( $\mbox{\textasciicircum}$ ) operator dealing with the odds of shared transitions. A memory structure has also been defined to keep track of past transitions in order to facilitate the backtracking without losing the initial computations, which may result in inconsistent states as well. The major contribution of this work can be seen in three aspects: first, defining an automaton to model full non-determinism along with probabilistic characterisation without making any distinctions between states and actions by which some of the previous formalism suffers; second, association of memory with the automata that preserves concurrency during backtracking; third, the implementation of the parallel joint operator to facilitate the concurrency. Although, the full probabilistic analysis of CPRA can be performed by applying the principles of measure theory on the traces of CPRA but it is reported as future work.