Concerns have been raised about the rate of progress at the Agratas gigafactory site in Somerset, with fears the project could be scaled back.