Written science explanations are central to learning and practicing science. However, there are two main challenges: first, students struggle to explain their ideas and use supporting data appropriately. Second, it is not realistic for teachers to provide real-time, comprehensive feedback to each student. We will present how PyrEval, a natural language processing (NLP) technology, provided students with timely, personalized, automated feedback. We will discuss opportunities and challenges for using automated feedback in classrooms.