Attack Speed does not affect your spell casting speed at all,in any way. It only improves the physical attack speed.

So why am I asking you as a caster to read this? Well Attack Speed also have the quality that it reduces the time the cast animation take to finish!

So? Well this means; even some of your instant casts have an animation that take time to complete. You cannot start to cast a spell (or do any action except moving) before the animation of the previous one is done. So even if your cast time is instant, the casting animation might not be.

You should therefore:
1. Learn and plan when/how to use your spell casts wisely, with the shortest animation first if no other advantages/disadvantages come from that choice of sequencing the spells.
2. Simply know of this fact and take it into consideration when getting a new gearset. If two items are alike, this can be a decisive factor in your choice.

Using the Attack Speed boost well, will allow you to fit the maximum number of casts into the timeslots you have. This also goes for non caster classes, and their abilities of course.