Htydjtk

If $p$ is a prime then the zeta function for an algebraic curve $V$ over $mathbb{F}_p$ is defined to be
$$zeta_{V,p}(s) := expleft(sum_{mgeq 1} frac{N_m}{m}(p^{-s})^mright). $$
where $N_m$ is the number of points over $mathbb{F}_{p^m}$.

I was wondering what is the motivation for this definition. The sum in the exponent is vaguely logarithmic. So maybe that explains the exponential?

What sort of information is the zeta function meant to encode and how does it do it? Also, how does this end up being a rational function?

The definition using exponential of such an ad hoc looking series is admittedly not too illuminating. You mention that the series looks vaguely logarithmic, and that's true because of denominator $m$. But then we can ask, why include $m$ in the denominator?

A "better" definition of a zeta function of a curve (more generally a variety) over involves an Euler product. The product will be over all points $P$ of $V$ which are defined the algebraic closure $overline{mathbb F_p}$ (this isn't exactly true, see below). Any such point has a minimal field of definition, namely the field $mathbb F_{p^n}$ generated by the coordinates of this point. We shall define the norm of this point $P$ as $|P|=p^n$. Then we can define
$$zeta_{V,p}(s)=prod_P(1-|P|^{-s})^{-1}.$$
(again, this is not quite right) Why would this definition be equivalent to yours? It's easiest to see by taking the logarithms. Then for a point $P$, the logarithm of the corresponding factor of the product will contribute
$$-log(1-|P|^{-s})=sum_{k=1}^inftyfrac{1}{k}|P|^{-ks}=sum_{k=1}^inftyfrac{1}{k}p^{-nks}=sum_{k=1}^inftyfrac{n}{nk}(p^{nk})^{-s}.$$
In the last step I have multiplied the numerator and the denominator by $n$, because the point $P$ contributes precisely to numbers $N_{nk}$, since $P$ is defined over fields $mathbb F_{p^{nk}}$.

But we see a problem - this way, we have counted each point $n$ times because of $n$ in the numerator. The resolution is rather tricky - instead of taking a product over points, we take a product over Galois orbits of the points - if we have a point $P$ mininally defined over $mathbb F_{p^n}$, then there are exactly $n$ points (conjugates of $P$) which we can reach from $P$ by considering the automorphisms of $mathbb F_{p^n}$. If we were to write $Q$ for this set of conjugates, and we define $|Q|=p^n$, then repeating the calculation above we see that we always count $Q$ $n$ times - which is just right, since it consists of $n$ points! Thus we arrive at the following (this time correct) definition of the zeta function:
$$zeta_{V,p}(s)=prod_Q(1-|Q|^{-s})^{-1},$$
the product this time over Galois orbits.

Apart from being (in my opinion) much better motivated, it has other advantages. For instance, from the product formula it is clear that the series has integer coefficients. Further, it highlights the similarity with the Riemann zeta function, which has a very similar Euler product. Both of those are generalized to the case of certain arithmetic schemes, but that might be a story for a different time.

As for your last question, regarding rationality, this is a rather nontrivial result, even, as far as I know, for curves. If you are interested in details, then I recommend taking a look at Koblitz's book "$p$-adic numbers, $p$-adic analysis and zeta functions". There he proves, using moderately elementary $p$-adic analysis, rationality of zeta functions of arbitrary varieties.

As KConrad says in the comment, the proof of rationality actually is much simpler for curves than for general varieties. I imagine it is by far more illuminating than Dwork's proof as presented in Koblitz.

Exercise 4.8 of Enumerative Combinatorics, vol. 1, second
ed., and Exercise 5.2(b) in volume 2 give an explanation of
sorts for general varieties over finite fields. According
to Exercise 4.8, a generating function $exp sum_{ngeq 1}
a_nfrac{x^n}{n}$ is rational if and only if we can write
$$
a_n=sum_{i=1}^ralpha_i^n-sum_{j=1}^s beta_j^n, $$
for nonzero complex numbers $alpha_i$, $beta_j$
(independent of $n$). This is stronger than saying that
$sum_{n geq 1}a_nx^n$ is rational. Moreover, if the
variety $V$ is defined over $mathbb{F}_q$ and $N_n$ is the
number of points over $mathbb{F}_{q^n}$, then the solution
to Exercise 5.2(b) is a simple argument showing that $exp
sum_{ngeq 1}N_nfrac{x^n}{n}$ has integer
coefficients. It corresponds to partitioning the rational
points into their Galois orbits.