Capacity of the multilayer perceptron with discrete synaptic couplings

We study the multilayer perceptron with N discrete synaptic couplings. As a typical discrete set of values, we assume that synaptic couplings take 2L+1 values k/L (k=-L,...,L, L is an integer). Using the replica method, we study the space of solutions that implement prescribed P input-output relations. The property of the space of solutions is featured by two characteristic α≡P/N values; one is the point ${\mathrm{\alpha }}_{\mathrm{AT}}$ where the Almeida-Thouless instability takes place and the other is the point ${\mathrm{\alpha }}_{\mathit{S}}$ where the entropy S of the replica symmetry (RS) theory vanishes. When the number of hidden units K is infinity, we find that the order of ${\mathrm{\alpha }}_{\mathrm{AT}}$ and ${\mathrm{\alpha }}_{\mathit{S}}$ changes at L=4. For L≥4, the replica symmetry has to be broken with the finite entropy of solutions. For large L, we find that ${\mathrm{\alpha }}_{\mathit{S}}$ is proportional to (lnL${)}^{\mathit{x}}$, where x is very close to 1/2. The one-step replica symmetry breaking theory gives a smaller value for S than the RS theory does, but the difference is very small. For K=3, we find that ${\mathrm{\alpha }}_{\mathit{S}}$ becomes larger than ${\mathrm{\alpha }}_{\mathrm{AT}}$ when L≥2. We also study numerically the space of solutions for K=3 using the least action algorithm modified for discrete coupling J. We find that some drastic change of the space of solutions really takes place around ${\mathrm{\alpha }}_{\mathrm{AT}}$.