There is the time module for iptables:
iptables ... -m time --timestart TIME --timestop TIME --days DAYS -j ACTION
Use -p tcp -j REJECT --reject-with tcp-rst to pretend the port is closed (the default action for REJECT is an ICMP packet).
I'd also add another rule in front, allowing ESTABLISHED connections through if I wanted open connections to remain working.
Compared to solutions using scripting to perform certain actions at certain times it has the great advantage that this is static configuration, and there are unlikely to be any failure modes that would e.g. permanently block access.