Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Several issues persist - prevents using metagpt for anything serious. #1769

Open
myevolve opened this issue Mar 17, 2025 · 2 comments
Open

Comments

@myevolve
Copy link

myevolve commented Mar 17, 2025

After extensive testing running metagpt for ~100 hours using openrouter:anthropic/claude-3.7-sonnet:thinking, I've concluded the following:

  1. json parsing fails when documents/code grow larger than it can handle. Sometimes I was successful in instructing it to work in smaller chunks and it would actually do it, however, most times it would not and continue to fail and exit/crashes.
  2. Context limit being exceeded and it does not appear that the "middle-out" / compression features of openrouter are implemented to prevent this.
  3. No easily observable way to just resume when it crashes. Instead, it will try sometimes and rebuild the docs it has already built, even when I specify that the docs are there for it to review.
  4. Token usage is very high per api requests making using metagpt for any sizable project not optimal.

The project shows promise, but there are many other projects that seem to have managed to address these simple issues well. It appears that the activity around this project is limited and reduced as compared to the past. Are there any plans to get things up to speed? Do you need help?

@ODAncona
Copy link

there are many other projects that seem to have managed to address these simple issues well

Such as ?

@myevolve
Copy link
Author

I have been working extensively with both open source and closed source projects:

CLI/Web: aider, openhands (similar to devin) -- This project has come a long way and does a pretty good job.

IDE: Roo Code, Cline, Windsurf, Cursor -- My testing for these has been mainly around configuring autonomous coding and providing the least amount of assistance other than the upfront project specifications to see if they can figure things out and advance the test project.

Projects that still struggle with some of the issues I mentioned, but all show promise if they can move fast enough:
Ra-aid, Codename Goose, Metagpt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants