Original answer
So it makes sense to conclude that the potential everywhere outside is
zero too as work done in moving the charge is zero
This is not correct. It's ok to conclude the potential is constant outside the plates based on your argument, but not zero. That's because the work $W$ needed to move a particle of charge $q$ from $a$ to $b$ is $W = q\int_a^b \vec{E} \cdot d \vec{\ell} = - q \int_a^b \vec{\nabla} V \cdot d \vec{\ell} = -q\left(V(b) - V(a)\right)$ (where $\vec{E}$ is the electric field, $V$ is the electric potential, and $d\vec{\ell}$ is an infinestimal line element along the path from $a$ to $b$). As you can see from the last equation, the work needed to move a particle from point $a$ to point $b$ is given by the difference of the potentials at $a$ and $b$ (multiplied by the charge of the particle). $W=0$ implies $V(a)=V(b)$, not $V(a)=0$.
The potential is a constant $V_0$ on one side of the plates, and a different constant $V_0 + \frac{\sigma \ell}{2 \epsilon_0}$ on the other side of the plates. (Often we just take $V_0=0$ for convenience). Between the plates, the potential linearly extrapolates between these two values, since the field (negative gradient of potential) is constant between the plates.
Additional answer
Restating problem
Based on some discussion in comments, I realized the above answer did not address the OP's question. Here is my way of phrasing their question as I understand it and how I'd approach it.
The potential due to a charged disk of radius $r$ and charge density $\sigma$ at position $z$ along the axis of symmetry (where the disk is at $z=0$) is (http://hyperphysics.phy-astr.gsu.edu/hbase/electric/potlin.html)
$$
V_{\rm disk}(z; 0, \sigma, r) = \frac{\sigma}{2\epsilon_0}\left(\sqrt{z^2+r^2} - |z|\right)
$$
I think the OP may have missed this absolute value (which is needed by the reflection symmetry across the plane of the disk), which led to incorrect conclusions.
Then the potential for two parallel disks, one with density $\sigma$ at $z=0$ and one at $-\sigma$ at $z=\ell$ is
\begin{eqnarray}
V_{\rm parallel\ disks}(z; \sigma, \ell, r) &=& V_{\rm disk}(z; 0, \sigma, r) + V_{\rm disk}(z; \ell, -\sigma, r) \\
&=& \frac{\sigma}{2\epsilon_0} \left(\sqrt{z^2 + r^2} - \sqrt{(z-\ell)^2 + r^2} - |z| + |z - \ell| \right)
\end{eqnarray}
We also know the potential for two parallel plates with densities $\pm \sigma$ of infinite size (at $z=0$ and $z=\ell$) is
$$
V_{\rm infinite\ plates}(z; \sigma, \ell) =
\begin{cases}
0, z< 0; \\
-\frac{\sigma z}{\epsilon_0}, 0 < z < \ell \\
-\frac{\sigma \ell}{\epsilon_0}, z > \ell
\end{cases}
$$
Now, we expect that the limit of infinitely large parallel disks should match the case of infinite parallel plates, up to an overall offset $V_0$. But what precisely does this mean?
Taking the limit, wrong way
One thought is that we want to understand the potential far away from the plates. So lets take $z \rightarrow \infty$ first. Then
$$
\lim_{z\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) = \lim_{z\rightarrow \infty} \frac{\sigma}{2\epsilon_0} \left(\sqrt{z^2 + r^2} - \sqrt{(z-\ell)^2 + r^2} - |z| + |z - \ell| \right) = 0
$$
By the same logic, we also have
$$
\lim_{z\rightarrow -\infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) =0
$$
However, this doesn't match our expectation of $V_{\rm infinite\ plates}(z; \sigma, \ell)$, which should have two different values as $z\rightarrow +\infty$ and $z\rightarrow-\infty$. What is going on?
We want to look at both $z$ and $r$ becoming large. Here, we took the limit $z\rightarrow \infty$ for fixed $r$, and found $V=0$ everywhere, since the potential is zero in every direction as you move away from the plates at finite $r$. But that isn't really the situation we are interested in, since we cannot get infinitely far away from infinite parallel plates in directions parallel to the plates.
In other words, the limits $z\rightarrow \infty$ and $r\rightarrow\infty$ do not commute. The reason there is a subtlety in this case is that the limit $r\rightarrow\infty$ changes the asymptotic boundary conditions; for finite $r$ you can get infinitely far away from the plates in any direction, but for infinite $r$ you cannot.
Instead, we should first take the limit $r\rightarrow\infty$ for fixed $z$, in order to guarantee we stay at a fixed distance from the plates as they become larger. Then, we can take the limit $z\rightarrow\infty$ safely.
Taking the limit, correctly
What we should really do is take the limit $r\rightarrow\infty$ for fixed $z$. Physically this corresponds to focusing in "nearby" the plates, then making the plates bigger and bigger, so that the edges of the plates become less and less important.
We expect that in this limit, up to an overall constant offset in potential $V_0$ (which does not affect any physical results), that the parallel disks should give us the same potential as the infinite parallel plates.
$$
\lim_{r\rightarrow \infty} \left(V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0\right) = V_{\rm infinite\ plates}(z; \sigma, \ell)
$$
Let's check this expectation. We start by evaluating the limit $r\rightarrow\infty$
\begin{eqnarray}
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0 &=& \lim_{r\rightarrow\infty} \frac{\sigma}{2\epsilon_0} \left((r - r) + \frac{z^2 - (z-\ell)^2}{2r} - |z| + |z-\ell|\right) + V_0 \\
&=& -\frac{\sigma}{2\epsilon_0} \left(|z| - |z-\ell|\right) + V_0
\end{eqnarray}
where we used the expansion $\sqrt{z^2+r^2}=r+\frac{z^2}{2r} + \cdots$ valid for large $r$. Here, we see those crucial absolute value signs appear, which will be very relevant for matching with $V_{\rm infinte\ plates}.$
We can handle the absolute values in cases.
Case 1: $z > \ell$
If $z > \ell$, then both $z$ and $z-\ell$ are positive, so $|z|-|z-\ell|=\ell$. Then
$$
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0 = -\frac{\sigma\ell}{2\epsilon_0} + V_0 \ \ (z>\ell)
$$
To match with the parallel plate case in this range, we should take
$$
V_0=-\frac{\sigma \ell}{2\epsilon_0}
$$
That gives the expected result
\begin{eqnarray}
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0 &=& -\frac{\sigma\ell}{\epsilon_0} \\
&=& V_{\rm infinite\ plates}(z; \sigma, \ell), \ \ (z > \ell)
\end{eqnarray}
Case 2: $0 < z < \ell$
If $0 < z < \ell$, then $z>0$ but $z-\ell<0$ so $|z|-|z-\ell|=z+z-\ell=2z-\ell$. Then
\begin{eqnarray}
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0 &=& -\frac{\sigma}{2\epsilon_0} \left(2z - \ell\right) + V_0\\
&=& -\frac{\sigma z}{\epsilon_0} - \frac{\sigma\ell}{2\epsilon_0} + V_0 \\
&=& -\frac{\sigma z}{\epsilon_0}
\end{eqnarray}
where the last two terms cancel because of our choice of $V_0$.
So we've shown
\begin{eqnarray}
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0 &=& -\frac{\sigma z}{\epsilon_0} \\
&=& V_{\rm infinite\ plates}(z; \sigma, \ell), \ \ (0 < z < \ell)
\end{eqnarray}
Case 3: $z < 0$
This case is similar to case 1, but now both $z$ and $z-\ell$ are negative, so $|z|-|z-\ell|=-\ell$.
Then
$$
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0= \frac{\sigma\ell}{2\epsilon_0} + V_0 = 0
$$
since we've taken $V_0=-\frac{\sigma \ell}{2\epsilon_0}$.
That means
\begin{eqnarray}
\lim_{r\rightarrow \infty} V_{\rm parallel\ disks}(z; \sigma, \ell, r) + V_0 &=& 0 \\
&=& V_{\rm infinite\ plates}(z; \sigma, \ell), \ \ (z < 0)
\end{eqnarray}
Summary
Therefore, we've explicitly verified our expectation the the potential on the symmetry axis of two parallel circular disks of radius $r$, in the limit $r\rightarrow\infty$, matches the potential of two infinite parallel plates.
We also saw we had to be careful to take the limit $r\rightarrow\infty$ first, and then the limit $z\rightarrow \pm \infty$, not the other way around. Mathematically, this subtlety arises because the boundary conditions for infinite plates are different than for finite plates. Physically, what we are saying is that the limit we are interested in is to zoom in very close to the center of the disks, and then make their radius much bigger than their separation so we can ignore effects of the edges of the disk. Moving very far away from the disks will not help us understand that situation.