Three months ago I saw that PyCharm shipped with a “Full Line Completion” plugin that “uses a local deep learning model to suggest entire lines of code”. These suggestions manifest as whole-line suggestions after you start typing and can be accepted with Tab. Essentially auto-complete for entire lines.

I decide to test this functionality. I started by writing import urllib3, created a new line, and then typed u and received a suggested completion for the line marked below with a dashed border. I was not impressed by the result:

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

Accepting this line would mean that any insecure requests made with urllib3 would not result in a user-visible warning. I didn't accept this suggestion and then began to instantiate a urllib3.PoolManager and what I feared would come next was confirmed:

import urllib3

urllib3.PoolManager(
    cert_reqs='CERT_NONE',

The suggestion offered to disable certificate verification (CERT_NONE) which would make every request made by the PoolManager susceptible to monster-in-the-middle (MITM) attacks. Accepting this code as-is would mean the program I am writing has a severe vulnerability. If I had accepted the prior suggestion too, then urllib3 would have no chance to warn the user about this mistake prior to productionizing this code.

Clearly something insecure is going on here, but for a CVE to be assigned we have to decide which software component is vulnerable. Does this behavior warrant a CVE at all? I am not sure which is unfortunate, without a security-angle to a bug report companies are less likely to prioritize reports.

I reported this behavior to JetBrains for “Full Line Code Completion” v253.29346.142 and clearly their support staff weren't certain whether this defect was a security vulnerability or not either. When I asked to publish a blog post about this behavior after they confirmed this report wasn’t a “direct security vulnerability” (which I agree with) but then was asked not to publicize my report and referred to PyCharm’s Coordinated Disclosure Policy so... which is it? Security vulnerability or not?

I ended up waiting the 90 days anyway and I didn't hear back with any substantive update from the development team. I double-checked again today using “Full Line Code Completion” v261.24374.152 and the behavior is identical, suggesting the same insecure code for both contexts.

This isn’t meant to be a specific dig at PyCharm or JetBrains, I have no-doubt that examples like this exist in every code generation model available. I don’t think using CVEs for this purpose is appropriate or helpful for users, either. But not prioritizing and addressing this behavior at the source means more work to mitigate the potential for insecure code to be accepted by users who are trusting what is offered to them by their IDE.

What do you think? I am interested in knowing your thoughts about this specific class of issue with code generation models.

Wow, you made it to the end!

• Share your thoughts with me on Mastodon, email, or Bluesky.
• Browse this blog’s archive of 183 entries.
• Check out this list of cool stuff I found on the internet.
• Follow this blog on RSS or the email newsletter.
• Go outside (best option)