A big conjecture which emerged from the math-physics interface is that the Gromov-Witten potentials of a Calabi-Yau threefold are generalized quasi-modular objects. In on-going work, joint with Aaron Pixton, we study the case of Picard rank 3 Abelian surface fibrations, where quasi-modular forms of Siegel type are expected to arise. We focus on the example of the banana manifold where we prove that the Gromov-Witten potentials satisfy the elliptic transformation law of a Siegel-Jacobi form for the E_{8} lattice. Moreover, a conjectural form is given in genus 0, and degree 1 over the base of the Abelian surface fibration.