Star Trek did it. Superman did it. Even Woody Allen traveled through time in one of his movies (bonus points for knowing which one). In short, we've been talking about traveling through time for a very long time.
James Gleick, who writes about science and its practitioners, travels back in time to the origins of people thinking and writing about time travel.
It's all on the table, from Jules Verne to the present day, from art to science to philosophy, in Gleick's book Time Travel: A History.
It includes an examination of what the author calls the porous boundary between science fiction and modern physics.