Is extending an abstract "base" implementation class an anti-pattern?

Since you can't "hide" your class's supertypes from your class's clients, its entire type hierarchy is part of its API, including the "base" class. Clients can store your implementation in a variable of the abstract "base" type. Good luck changing your supertype after that.

Since the "base" class does not define semantics of the class and is only extended to get access to certain implementation decisions, it's an implementation detail.

Hence, it makes implementation details part of your class's API which is strictly forbidden in software design.

True, you can make the "base" class package-private, but I'm personally not a big fan of package-private classes since they restrict my ability to change the package hierarchy.

If it is an anti-pattern, should you make you class contain the base class ("prefer composition over inheritance")? Won't it be confusing?

Despite my criticism of the "base" classes, they are convenient, pretty common, especially in older APIs (e.g. Swing) and in practice cause little harm.