Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in GpPolicyFetch() called in RelationBuildDesc() when the table is an external table #11244

Closed
hidva opened this issue Dec 3, 2020 · 1 comment · Fixed by #11248

Comments

@hidva
Copy link
Contributor

hidva commented Dec 3, 2020

Greenplum version or build

PostgreSQL 12beta2 (Greenplum Database 7.0.0-alpha.0+dev.14108.g57d5181d5d build dev) on x86_64-pc-linux-gnu, compiled by gcc (GCC) 6.5.1 20190307 (Alibaba 6.5.1-1 2.17), 64-bit compiled on Dec  3 2020 21:51:34 (with assert checking)

Step to reproduce the behavior

createdb tmp
nohup pgbench -c 1 -j 1 -f test.sql -n -T 3600000 tmp >/dev/null 2>&1 &
nohup pidstat -p ${QDPID} -r 1 >${QDPID}.pidstat.log 2>&1 &

The content of test.sql is:

select * from gp_toolkit.__gp_log_master_ext limit 1;

image

$ gcore ${QDPID}
$ strings core.${QDPID} | sort | uniq -c | sort -nr | head -n 16
1715633 encoding
1715626 command
1715625 quote
1715625 null
1715625 format
1715625 escape
1715625 delimiter
1715624 MASTER_ONLY
1715624 log_errors
1715624 is_writable
1715624 format_type
1715624 false
1715624 execute_on
1715624 cat $GP_SEG_DATADIR/log/*.csv
   2832 UTF-8
    841 ISO-8859-1

Root Cause

ExtTableEntry *e is allocated in CacheMemoryContext that will never be reset or deleted.

// RelationBuildDesc
if ((relation->rd_rel->relkind == RELKIND_RELATION && !IsSystemRelation(relation)) ||
    relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE ||
    relation->rd_rel->relkind == RELKIND_FOREIGN_TABLE ||
    relation->rd_rel->relkind == RELKIND_MATVIEW)
{
    MemoryContext oldcontext = MemoryContextSwitchTo(CacheMemoryContext);
    relation->rd_cdbpolicy = GpPolicyFetch(targetRelId);
    MemoryContextSwitchTo(oldcontext);
}
GpPolicy *
GpPolicyFetch(Oid tbloid)
{
    if (rel_is_external_table(tbloid))
    {
        ExtTableEntry *e = GetExtTableEntryIfExists(tbloid);
@JunfengYang
Copy link
Contributor

JunfengYang commented Dec 4, 2020

Feel free to open PR fix the issue. Thanks!

hidva added a commit to hidva/gpdb that referenced this issue Dec 4, 2020
kainwen-zz pushed a commit that referenced this issue Jan 8, 2021
Fix memory leak in GpPolicyFetch() called in RelationBuildDesc() when the table is
an external table. This commit fixes github issue #11244.
Also, remove the Assert(CurrentMemoryContext != CacheMemoryContext).  I still think
we should add this assert, but the reality is that there are many places in the code that
overuse the CacheMemoryContext, so we have to remove this assert.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants