I presume that the first assemblers were written in machine code, because as you correctly guess, nothing else was available back then.
Today, however, when a brand new CPU architecture comes out, we use what is known as a Cross-Compiler, which is a compiler that produces machine code not for the architecture on which it is running, but for a different architecture.
(As a matter of fact, as I am sure you will find out later on in the book you are reading, there is absolutely nothing which makes a compiler inherently more suitable for producing machine code for the architecture on which it is running than on any other architecture. It is just a matter of which architecture you, as the creator of the compiler, are going to target.)
So, today it is even possible (at least in theory) to create a brand new architecture and have high-level language compilers natively running on it (compiled on other architectures using cross-compilers) before you even have an assembler for that architecture.