IEnumerator contains MoveNext(), Reset() and Current as its members. Now assume that I have moved these methods and property to IEnumerable interface and removed GetEnumerator() method and IEnumerator interface.
Now, the object of the class which implements IEnumerable will be able to access the methods and the property and hence can be iterated upon.
IEnumerator interface solve those problems?

An iterator contains separate state to the collection: it contains a cursor for where you are within the collection. As such, there has to be a separate object to represent that extra state, a way to get that object, and operations on that object - hence IEnumerator (and IEnumerator<T>), GetEnumerator(), and the iterator members.
Imagine if we didn't have the separate state, and then we wrote:
var list = new List<int> { 1, 2, 3 };
foreach (var x in list)
{
foreach (var y in list)
{
Console.WriteLine("{0} {1}", x, y);
}
}
That should print "1 1", "1 2", "1 3", "2 1" etc... but without any extra state, how could it "know" the two different positions of the two loops?
See more on this question at Stackoverflow