You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After extensive testing running metagpt for ~100 hours using openrouter:anthropic/claude-3.7-sonnet:thinking, I've concluded the following:
json parsing fails when documents/code grow larger than it can handle. Sometimes I was successful in instructing it to work in smaller chunks and it would actually do it, however, most times it would not and continue to fail and exit/crashes.
Context limit being exceeded and it does not appear that the "middle-out" / compression features of openrouter are implemented to prevent this.
No easily observable way to just resume when it crashes. Instead, it will try sometimes and rebuild the docs it has already built, even when I specify that the docs are there for it to review.
Token usage is very high per api requests making using metagpt for any sizable project not optimal.
The project shows promise, but there are many other projects that seem to have managed to address these simple issues well. It appears that the activity around this project is limited and reduced as compared to the past. Are there any plans to get things up to speed? Do you need help?
The text was updated successfully, but these errors were encountered:
I have been working extensively with both open source and closed source projects:
CLI/Web: aider, openhands (similar to devin) -- This project has come a long way and does a pretty good job.
IDE: Roo Code, Cline, Windsurf, Cursor -- My testing for these has been mainly around configuring autonomous coding and providing the least amount of assistance other than the upfront project specifications to see if they can figure things out and advance the test project.
Projects that still struggle with some of the issues I mentioned, but all show promise if they can move fast enough:
Ra-aid, Codename Goose, Metagpt
After extensive testing running metagpt for ~100 hours using openrouter:anthropic/claude-3.7-sonnet:thinking, I've concluded the following:
The project shows promise, but there are many other projects that seem to have managed to address these simple issues well. It appears that the activity around this project is limited and reduced as compared to the past. Are there any plans to get things up to speed? Do you need help?
The text was updated successfully, but these errors were encountered: