The museum occupies the Mudéjar Pavilion designed by Aníbal González and built in 1914. It served as an art pavilion, the Pabellón de Arte Antiguo, for the Ibero-American Exposition of 1929, when Aníbal González had the opportunity to design several additional permanent buildings for the plaza. The exterior is ceramic over brick, and has three doors with archivolts adorned with glazed tiles.

The building originally consisted of two stories over an aeration chamber (necessary because of the humidity). In the 1960s, the main floor, originally over 12 metres high, was divided in two, with an intermediate level added by architect Delgado Roig.

In 1972, as part of the preparations for the current museum, the various stories of the building were connected by a grand spiral staircase designed by José Galnares Sagastizábal.

The museum was established by a decree of 23 March 1972, initially as a section of the Museum of Fine Arts of Seville. It opened its doors to the public 4 March 1973.

For the next seven years, the museum fell under the responsibility of the Ministry of Education and Science, but the city government occupied more than half of the poorly maintained building. Neither institution took full responsibility.

This situation led to a series of temporary closures. In 1976, the electric bill could not be paid, and the museum had to be shut until it could get power again. In 1979, heavy rains damaged the interior.

On 26 March 1980, the City Council agreed to allow the Ministry of Education and Science to use the whole building, for as long as it was used for the museum, on the condition that the Ministry would suitably restore it. On this basis, the Ministry hired architect Fernando Villanueva Sandino.

The resulting restoration converted the primitive aeration chamber into a new lower-ground floor. The museum reopened 18 October 1984; at that same time the new Andalusian Autonomous Government took over the museum, along with other museums in Andalusia that had been operated by the central government of Spain.

In 1990 the main floor was further fitted out for temporary exhibitions, and in 1994 the Díaz Velázquez Collection was permanently installed, with a little over a third of the building remaining for temporary exhibitions.